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A CONVERSATION WITH GLENN REEVES 2] 
by Jack J. Woebr 
Pathfinder was a decade-long program of robotic exploration of the planet Mars. Glenn Reeves, the 


mission’s “Flight Software Cognizant Engineer,” chats with Jack Woehr about Pathfinder and what's 
involved in writing software for extraterrestrial exploration. 


LINUX, REAL-TIME LINUX, & IPC 32 
by Frederick M. Proctor 

When dealing with real-time systems, IPC overhead becomes important. Fred examines two of the 

best IPC mechanisms available under Linux—FIFO and shared memory. 


INSIDE WINDOWS NT SYSTEM DATA 40 
by Sven B. Schreiber 
Sven untangles the mysterious Windows NT NtQuerySystemInformation function. 


EXTENDING THE PALMOS 50 
by Greg Winton 

Packaging code into shared libraries can enhance code reuse and reduce in-memory footprints. 

Greg takes a look at the Palm shared library model and presents a shared library you can use. 


THE POWERTV OPERATING SYSTEM 56 
by Morgan Woodson 

The PowerTV operating system is designed for set-top boxes and hybrid fiber/cable networks. 

Morgan provides an overview of the OS, then presents a weather/traffic camera viewer application. 


THE BERAYS RAY TRACER 62 
by Regan Russell 

Regan presents BeRays, an object-oriented ray-tracer application that makes it easy to plug in new 
rendering ideas. 









DESIGNING THE LEGOS MULTITASKING OPERATING SYSTEM 66 

by Markus L. Noga 

LegOS, the tiny, yet powerful operating system Markus built as an alternative to the RCX firmware, Photographs of Glenn Reeves by Pat Johnson Studios. 
supports features such as preemptive multitasking, dynamic memory management, POSIX All JPL/NASA project images courtesy of 
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eo 7ne i aL) Jbed, a small, fast Java Virtual Machine for embedded real-time systems, includes a complete 
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Interviews with Larry Wall, Brewster Kahle, MACHINE LEARNING & AGENT-BASED COMPUTING 88 
Guy Kawasaki, Tim OReilly, by Zhimin Ding and Li Liu 


Tom Christiansen, and more... Zhimin and Li examine the application of machine-learning technology to control software 
agents, and present MLEngine— a general-purpose AI engine with real-time learning capability. 
For more see 


http://www.ddj.com/technetcast/ 
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TRENDS IN HIGH-TECH HIRING PRACTICES §2 
by Reginald B. Charney 

Where should you focus your career-development efforts? Reg analyzes 

technical employment listings to find the answer. 


THE WALDEN PROGRAMMER ANALYST APTITUDE TEST $6 
by Bruce Winrow 

The Programmer Analyst Aptitude Test from Walden Testing has become an 

essential tool for evaluating high-tech hires. 





GAUGING CORPORATE CULTURE $16 
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CAREERS IN SYSTEM ADMINISTRATION $20 ooo 
by Barbara Dijker EDITORIAL 8 
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and organizational skills, are in demand in the high-tech job marketplace. 
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XML AND CORBA 98 NEWS & VIEWS 18 
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( 0 | T M 1 § by Nick Baran and Amy Lincicum 
SWAINE’S FLAMES 144 
PROGRAMMING PARADIGMS 103 by Michael Swaine 
by Michael Swaine 
According to Arthur C. Clarke, any sufficiently advanced technology looks like RESOURCE CENTER 
magic. Michael looks at some of the magical moments that are shaping the world. As a service to our readers, source code and 
related files, and author guidelines are available 
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Real-world data management solutions 
are typically more complex when one 
examines the pieces, than initially 
recognized by the majority of database 
programers. All software projects are 
complex puzzles comprised of many 


details, most of which are data-related. 


Often today’s “DBMS” solutions sacrifice 
the speed or control essential for a 


competitive application. 


c-tree Plus®, by FairCom, has been the 
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“T Said Turn 
RIGHT. Fool!” 





The “old” Hertz NeverLost GPS. 





EDITORIAL 


minefield. Dismissed by some as “a Y2K preseason game,” EOW rollover was billed by others 

as a Y2K warm-up that threatened the accuracy of global positioning systems (GPS) around the 
world. In a nutshell, the EOW rollover problem involved GPS receivers— not the satellites or 
transmitters — that might have thought the date was January 6, 1980, as GPS time rolled over from 
August 21 to August 20, 1999. 

According to the U.S. Coast Guard Navigation Center (http://www.naven.uscg.mil/gps/geninfo/ 
y2k/), the timescale origin of GPS time started at midnight January 5-6, 1980 (or Julian Day 
2,444,244.500). Since a GPS cycle is 1024 weeks (or 7168 days), our first GPS rollover was due to 
occur at Julian Day (2444244.5+7168)=2,451,412.5—00:00:00 on August 22, 1999, in other words. 
Putting this another way, that’s 7168/365.25=19.6249 years, or 1980+19.625=1999.638. No 
matter how you count it, it still comes out to August 22, 1999. (For details, see Richard 
Langley’s excellent description of the EOW phenomenon is “The GPS End-of-Week 
Rollover” at http://www.gpsworld.com/0299/1198innovation. html.) 

The upshot of all this was the potential for calamity, since millions of GPS receivers 
are being used by business, military, and consumers for everything from navigation 
and telecommunications, to finance and network computing. Of course, there 
shouldn’t have been any problem for GPS receivers with updated Y2K/EOW- rollover- 
compliant firmware. 

EOW rollover panic wasn’t rampant. No, unlike Y2K, there were no conferences, 
books, consultants, newsletters, web sites, or CNN talking heads trying to scare you out 
of a few bucks. Yes, there were scattered reports of frantic bass fishermen who depend 
on GPS to pinpoint secret fishing holes. As it turned out, there was little need to worry, at 
least in the U.S. where virtually no rollover-related problems were reported. That wasn’t 
the case in Japan, however, where hundreds of people were confused and 
inconvenienced, as screens on car-navigation systems went blank or were filled with 
random characters. At least we know where all the nonEOW-compliant receivers were. In 
short, GPS EOW rollover came and went rather quietly— this time around. 

I suppose you'd call me a Jonny-come-lately when it comes to this GPS stuff. Gee, 
even my sister, who is an archeologist, is way ahead of me. Faith has been using GPS for years to 
identify and locate historical sites in the Negev Desert. (She reports via e-mail, by the way, that she 
didn’t experience any problems with EOW rollover.) In fact, she’s on to her second- or third- 
generation GPS receiver, and insists that they are as necessary to her work as, well, a shovel and a 
pith helmet. 

The extent of my GPS experience, on the other hand, is with Hertz rental cars that have NeverLost 
In-Car Navigation Systems installed. NeverLost (http://www.hertz.com/serv/us/prod_lost.html), which 
uses GPS, dead-reckoning, and map-matching technology, is a private-label version of Magellan’s 
proprietary 75ONAV vehicle-navigation and driver- information system (http://www.magellangps 
.com/). More than 50,000 Hertz cars currently come with Magellan hardware and Navigation 
Technologies (http://www.navtech.com/) databases covering more than 50 U.S. cities. 

The Ford Taurus I was driving, for instance, had the entire database for California online. Using a 
clunky interface, you pressed the cursor pad to select the destination city from a long list of cities, 
then the street, and finally the address you want to drive to. The system takes a few seconds to 
compute the best route, then displays on-screen icons and issues audible voice commands to get you 
going in the right direction. As you approach a required turn, the system alerts you with a beep about 
a block prior to the actual junction, followed by a voice announcement, such as “Right Turn Ahead.” If 
you miss the turn, you need only press Enter and the system computes a new route for you (and, 
mercifully, doesn’t add the word “...fool.”) The Hertz web site and TV commercials are now hawking 
NeverLost units with a new cursor pad and improved interface— specifically, a QuickSpell feature that 
lets you go more directly to a city or street by selecting the first letter of the name, rather than scrolling 
down a long list. However, the Hertz cars I’ve recently rented still have units with the old interface. 

Okay, I admit that until I actually used one, I thought systems like Hertz’s NeverLost GPS 
were nothing more than marketing gimmicks. But they truly are a valuable application of 


t was with great relief that we safely navigated this summer’s end-of-week (EOW) rollover 


technology that just happens to be fun. Even with the old NeverLost interface, in-car navigation 


is relatively easy to use, and certainly much more convenient (and safer) than pulling over to the 
curb and spreading out a map. And as for the EOW rollover interfering with GPS, don’t worry— 


it won't come again for another 19 years. 


Jonathan Erickson 
editor-in-chief 
jerickson@ddj.com 
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Building setups that include third party technologies is a constant battle. 


InstallShield Professional 2000 is the peace plan. 


INSTALLSHIELD OBJECTS 
make it dramatically simpler 
to include and deploy 
third-party technologies. 


DEPENDENCY MANAGER 
automatically scans your app to 
identify required DLL’s 
and offers choices on whether 
(and where) they 
should be incorporated. 


COMPONENT 
INSTALL/UNINSTALL 
deliver individual features, 
applications or suites 
that can be easily changed, 
removed, or reinstalled 
by the end user. No need for 
custom DLLs for Uninstallation. 


EVENT-BASED SCRIPTING 
improved script management 
Structure reduces the amount of 
code necessary to create setups, 
so you have more time to 
customize them. 


The code war is over. The newest version of the industry standard for installation development is here: InstallShield Professional 2000 
— delivering breakthrough usability and productivity for developers who create setups of any size or complexity, for any Windows platform 
— including Windows 2000. As a developer, your most precious commodity is time. And you'll save plenty of it with the powerful new 


features in InstallShield Professional 2000; like InstallShield Objects, which take the grunt work out of building installations by making it 


about InstallShield Professional 2000, download a trial version, or purchase it immediately, ns Q le . 


and start using the newest version of the industry standard in setup authoring today. $0 FIWARE C0 RPORATI 0 N 


easy to support established third-party technologies. So don’t waste another minute fighting 


the installation war. Visit www.installshield.com/peace or call 800-969-7724 to learn more 
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Programmer Aptitude Testing 
Dear DDJ, 
Readers interested in the “Wolfe pro- 
gramming aptitude test” (see “Letters” DD/J, 
September 1999) might find the history of 
the test interesting as well. Our Aptitude 
Assessment Battery Programming test was 
developed by the late Dr. Jack M. Wolfe, 
Professor Emeritus of Computer Science 
at City University of New York. Dr. Wolfe 
was one of the pioneers of computer sci- 
ence in the 1960s and developed his test 
to replace the old IBM test which he 
thought had major deficiencies. The test 
is a simulation of what a programmer is 
required to do. It is virtually always ac- 
curate in predicting success as a pro- 
grammer. It measures not only logical abil- 
ity and problem solving ability, but such 
qualitative factors as documentation and 
annotation ability, whether a person is bet- 
ter suited for maintenance or creative pro- 
gramming, whether and how well the can- 
didate desk checks his work and whether 
he is a faster or slower than average per- 
former. It is available in several languages, 
in Braille, and in a left-handed version (no 
kidding). The scoring process took Dr. 
Wolfe over seven years to perfect and val- 
idate. The CIA used it for over 15 years 
to assess its programmer applicants and 
validated it; so have thousands of other 
organizations. 

Stephen Silver 

Walden Personnel Testing & Training 

ssilver@waldentesting.com 


DDJ responds: DDJ readers attending the 
Software Development ’99 East Conference 
(http://www.sdexpo.com/) in Washington, 
DC (November 10-11) can take the Walden 
Programmer Analyst Aptitude Test at no 
charge in a special program sponsored by 
DDjJ. If you’re interested and are planning 
on attending the conference, sign up by 
sending e-mail to editors@ddj.com. 


Cross-Platform Design 

Dear DDJ, 

I read with interest Bob Krause’s “Cross- 
Platform Design Strategies” (DD/, June 


10 


1999). The issue is not as simple as Bob 
makes it. While his goals are worthy, the 
thread classes he describes actually fall 
into some of the problems he is trying to 
avoid. The problem is that, in general, OS 
differences just cannot be cleanly ab- 
stracted without losing functionality. 

In the thread case, right at the start there 
is the issue of preemptive thread switch- 
ing. His thread classes choose to explicit- 
ly disable preemptive switching. This is a 
lowest common denominator decision. That 
is not to say that it was a bad decision. 

The other issue was the block call. By 
not fully supporting the ability to block 
with a timeout the abstraction is lost. Ei- 
ther the programmer has to make do with- 
out that functionality or has to make OS- 
specific code to handle the differences. 

These abstractions are still the way to 
achieve a cross-platform application but 
they are not truly generic since they in- 
clude assumptions about the functionali- 
ty that is required and these should be 
made explicit. 

Tim Bond 

tbond@phios.com 


A Hard Day’s Night 
Dear DDJ, 
In “News & Views” (DDJ, September 1999), 
we were told that Pitney Bowes reckons 
that the average U.S. worker receives 200 
e-mails each day and is interrupted by fax, 
phone, or e-mails about every 10 minutes. 
I don’t think so! A little simple arithmetic — 
no computer needed — shows that even if 
the average U.S. worker works 24 hours 
each day and receives 200 e-mails, then 
the average interval is 7.2 minutes— just 
for e-mails. Even programmers don’t work 
24 hours every day and I doubt that the 
average worker receives 200 e-mails each 
day. Perhaps we should be a bit skeptical 
of reports like these from Pitney Bowes? 
They aren’t perhaps, trying to sell some- 
thing, are they? 

Mike Pidd 

M.Pidd@lancaster.ac.uk 


Electronic Music 

Dear DDJ, 

When I was about 17 years old, circa 
1961, I cobbled together a bunch of con- 
traptions for the French department lan- 
guage lab that sound a lot like the ma- 
chines Al Stevens described in his “C 
Programming” column (DD/, September 
1999)— raw materials were some Phillips 
reel-to-reel tape recorders, 117-VAC re- 
lays, and switch boxes. Students played 
a reel on the machine in front of them, 
another hidden machine with a 20 sec- 
ond (fixed) endless loop recorded the out- 
put on one track. Students could turn a 
rotary switch to stop the playback and 
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hear the loop, another position to record 
their own rendition on a different track 
of the loop, flip back and forth until they 
were happy with it, and go back to the 
main machine. Not bad for the time. 

The department head published this 
nonsense in, I think, “The Journal of 
Catholic Education,” about 1962-63. If, as 
it seems, nameless_agency.gov had pri- 
ority, we will have to print a retraction. 
Please advise. 

Dugald A. Taylor, M.D. 

Wichita, Kansas 


So What’s Wrong with COBOL? 
Dear DD/, 
Have you also noticed that the simple 
things we did in Cobol (or Fortran, or BA- 
SICA, or whatever) have become next to 
impossible today? I just want to access my 
COM port with VB6.0 Learning Edition or 
update a small text file on a server (with- 
out needing Cold Fusion), or stick a few 
debugging “beeps” into Visual C++. I put 
a clever little conversational CGI program 
on my ISP’s server to do a calculation (like 
good old CICS). It was simple and well 
behaved on my PC, but because I com- 
piled it with a 16-bit compiler, it brought 
the entire server crashing down. My ISP 
thinks I’m an idiot. ’m not.) Turns out 
Microsoft has a bug in its IIS server that 
can’t deal with an older MS exe file. I have 
a lot of respect for mainframe hardware 
and software. You never get a hardware 
error and when you get a software error, 
there is a clear cause that you can deter- 
mine and correct on your own, most of 
the time. And you can’t crash a mainframe 
(by yourself). They only reboot (called 
IPL) for a damn good reason. Yes, Cobol 
is wordy and relatively unsophisticated, 
but the beauty is that no time was ever 
lost getting some simple customer file to 
print or a badly needed summary total 
flashed on a screen. It never seemed that 
important in a business setting to run a 
movie or marquee some stupid message 
like an old teletype. Is this part of the 
“dumbing down” trend? 

Chris Clement 

micrometer@compuserve.com 


Dear DDJ, 

Could you pass along to Philip Hendrick- 
son that I do program in Cobol and C and 
that I’m not working on Y2K issues, that 
our major application is written in Cobol — 
that’s right Cobol— and doctors and hos- 
pitals use our application everyday and will 
continue to use our application because it 
is reliable and runs on a reliable operating 
system “AIX” by IBM. So since I write small 
utilities in C and major applications in Cobol 
I guess that makes me a real programmer 
who likes Cobol? Oh by the way we use 
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(continued from page 10) 
MicroFocus Cobol for our billing soft- 
ware and RM Cobol for our charting 
software. 

Richard Shirey 

gizzmaxx@cobweb.net 


Dear DDJ, 
Everyone has been saying “Cobol is dead” 
for years now. I work for the 32nd largest 
credit card processing bank. It is a private 
company. There are only 250 of us, in- 
cluding the HR and receptionists. But I 
make good money and am happy. I am 
over 60— programming (in Cobol on Tan- 
dem— oops Compaq) for over 15 years. 
My degree is in business. 

In addition to the processing switch 
(millions of transactions per month), we 





do license work for large companies 
whose names you would know, refunds 
for a home improvement, multiple ap- 
plications for a Texas-based company. 
All they have in common is large, com- 
plex systems that work well, run on box- 
es that are 24/7 transaction processors 
with fault tolerance required— and they 
are written in Cobol. We also use PC 
workstations, telecommuting, NT servers, 
Internet mail, and all the other “good” 
stuff that real programmers write. But 
they are tools. 

Even a nonprogrammer can support 
self-documenting code simply by read- 
ing the source. Then when a new en- 
hancement is required, or an “undocu- 
mented feature” is discovered (read 
some real-time employee tried to do 


J Build your own 


_! Shareware 


Uf XRT Widgets for Motif 


Professionals know there's only one answer to Motif 
interface development: XRT Widgets from KL Group. 


XRT/table — Multi-purpose lists, tables and forms 


Data may be supplied programmatically, read from a file, 

entered by users, or supplied at run-time using a callback 
Supports table sizes of up to 2 billion rows by 2 billion columns 
Advanced Widget-in-Cell technology lets you efficiently manage 


widgets and pixmaps 


Includes label widget that supports rotated text and PostScript output 


Fast preview scrolling 
e Royalty-free 


XRT PDS also includes: 








XRT/graph — Wide variety of 2D charts and graphs 
XRT/3d — Sophisticated 3D charts and graphs 
XRT/gear — Essential add-on widgets and utilities 
XRT/field — Effortless data-entry and validation 


XRT supports most major UNIX platforms, VMS, and Windows via Interix and NuTCRACKER. Integrates 
with major GUI builders including UIM/X, Builder Xcessory, TeleUSE, View.h++, X-Designer and XRunner. 





KL Group Inc. 
1-888-361-9161 
Tel: 416-594-1026 
Fax: 416-594-1919 


ee . £h Pe Ce Oe Be 
Software Development Productivity 


FREE XRT Widget Tips and Progress Bar at: 
www.kligroup.com/widgets 







KL Group Europe 
Tel: +31 (0)20 679-95 03 
Fax: +31 (0)20 470 03 26 





XRT, XRT PDS, XRT/gecr, XRT/oraph, XRT/fild, XRT/table, XRT/d ond. XRT/qroph or trademarks of KL 
Group Inc. All other products are trademarks or registered trademarks of their respective companies. DDOOO 
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something we didn’t think of in QA and 
got a weird response) a programmer 
makes the changes. 

All you “real” programmers keep 
trucking. I enjoy my sailboat, have a 
wife, kids, grandkids, two dogs, and a 
cat that thinks he’s a dog. While you are 
stressed with three cell phones, two 
beepers, a laptop, and who knows what 
else, I will continue to code in Cobol 
until I am 90, telecomputing from my 
bed in the nursing home. 

GeorgeVol@aol.com 


Porting to CE 

Dear DDJ, 

After reading the article “Porting Com- 
munications Software to Windows CE”, by 
Oliver Deiner (DD/J, July 1999), I have 
some feedback and solutions to a few of 
the points he raised. 

Regarding overlapped I/O support in 
CE: It is true that the feature is not sup- 
ported directly by the file I/O functions 
(CreateFile, ReadFile, and so on), but it 
is indirectly supported for serial com- 
munications at the device driver level. 
The Microsoft Knowledge Base article 
Q175551 “INFO: Serial Communications 
on Windows CE HPC devices” discuss- 
es the exact details of the support pro- 
vided. 

Support for TAPI inbound connections 
has been the big complaint about CE 2.x 
and it has been publicly stated that the 
feature will definitely be available in the 
next major release of CE. See Doug Bol- 
ing’s article “Updated with New Kernel 
Features, Windows CE 3.0 Really Packs 
a Punch” (Microsoft Systems Journal, July 
1999). I wrote major portions of the Vi- 
sual C++ for Windows CE debugger so 
his comments with regard to remote 
communication are near and dear to my 
heart. Oliver’s criticisms are completely valid, 
but the easy solution is to use an Ethernet 
connection instead of the serial port. There 
is a hack to get this working in VCCE 2.0 
and it is fully supported with Platform Man- 
ager in the current version, VCCE 6.0. The 
slides and video from my 1999 CE Devel- 
oper’s Conference presentation “Tips and 
Tricks for Debugging with VCCE” are avail- 
able at http://events. microsoft.com/is- 
api/events/pages.asp?s= 63814&p=623 and 
give details on how to set this up. Thanks 
for an interesting and informative article on 
Oliver’s experience. It’s feedback such as 
this that will help improve future versions 
of both the OS and the development envi- 
ronment. 

Andrew Tucker 

andrewt@bsquare.com 
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You feel pulled in two directions 


On one hand, thanks to piracy and 
unauthorized use, you know you're losing 
one dollar for every dollar you make. 


On the other hand, you’re afraid you'll 
lose customers if you try to protect 
your applications. 


HASP® absolutely destroys 
that dilemma! 


HASP gives you outstanding protection, 
plus it’s extremely friendly to your end-users. 


In fact, National Software Testing Laboratories 
compared HASP to our major competitor and 
declared HASP Number One.* 


Twenty-five thousand developers have 
solved their dilemma by choosing HASP. 
Why don’t you? 

Call for your Developer’s 
Kit today! 


www.eAladdin.com 


Securing The Global Village 
USA 1-(800) 223-4277 
Intl. 972-3-636-2222 


Offices in: United States, Israel, Germany, United Kingdom, 
Netherlands, Japan and South Africa. Fifty-plus distributors 
serving more than 100 countries. 


HASP® is a registered trademark of Aladdin Knowledge Systems, Ltd. Copyright 1999 ¢ *Full copy of study available upon request. 
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Sign up here 
for e-mail 
newsletter! 


~~ SPREAD 


FarPoint’s Spread 3 
by FarPoint Technologies 


Use FarPoint's Spread 3 to create 
powerful database front-ends, manage 
the display and entry of two billion items. 
using the enhanced Virtual Mode, print 
reports using Print Preview, perform 
calculations, import/export Excel 97 files 
(32-bit versions), export HTML files (32-bit 
versions), sort data, support OLE Drag 
and Drop, or take advantage of its 
unparalleled cell-level formatting, 
including twelve built-in cell types. 
Unmatched power, flexibility, and speed 
combine to make Spread the perfect 
component for any application. 
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"Wasp F Bar Code 
_ ActiveX & DLL ~ | 


by Informatics 


e Supports all major bar code symbologies. 


Optional support for new 2D symbols 
(e.g. PDF417, MaxiCode, DataMatrix) 
e Render spec compliant bar codes to any 
device context 
© Two ActiveX controls fully support 
Drawing and Font DLLs and can be 
bound to a database. 
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~ TestTrack 5. User 


Installation Suites’ : 















by Wise Solutions 


: = Rule Pu T 7 
Wise Solutions introduces 

a new generation of a dah | 

versatile installation suites =e 

for a wide range of a emma 

developer needs with new features and enhance- 
ments that make installation scripting, patching, 
repackaging, and Web deployment a snap. 


by Seapine Software 


TestTrack is the fastest and 
most complete multi-user 
i bug tracking solution for 

2 New | | Windows 95/NT. Tracks bug _ 

: and feature requests, customers, users, test esnfidindlions and 
Version ¢| more, Advanced features include e-mail notifications, e-mail bug 
bay import, duplicate bug handling, release note generation, and 
much more. Distribute TestIrack’s standalone bug reporter to 
your customers to automate customer 
support. With all of its power, TestTrack 
remains the easiest bug tracking solution 
to use and maintain. 
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VTune Performance 


1 pkg.: : 
Enhancement Environment 


Supports < 
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by Faircom 


DOS ¢ WINDOWS e NT e UNIX 

e 0S/2 ¢ SUN e RS6000 © HP9000 
e MAC @ ONX @ LINUX @ SCO. 
This well known, highly-portable 
data management package has become 
established as the tool of choice for 
commercial development. Offering 
unprecedented data control, choose from 
direct low level access, ISAM level, or ODBC 
access with the FairCom Server. Single User, 
Multi User, or optional Client/Server, ANSI 
Standard. Full Source. 

No Royalties. 










by Intel 


The comprehensive solution for 
developers of high performance 
software for Intel® Architecture 
computers—from the people who 
design the processors. 


e VTune” Performance Analyzer 
e Intel C/C++ Compiler 

e Intel Performance Library Suite 
e Intel Performance Training Center. 
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SPF/SourceEdit v.2.5 
by Command Technology 


DevIrack’ 3.5 
by TechExcel 


DevTrack is the premier defect- and 
project-tracking tool that comprehensively 
tracks and manages all defect and develop- 
ment issues. DevTrack 3.5 adds exciting 
enterprise-level features: a definable work- 
flow; master/detail relationships; field-level 
security; and 





































Features include: ISPF/PDF command 
set, source colorization; ASCII or 
EBCDIC character set; Variable or 
Fixed records; record lengths to 64K; 
file sizes to 100M; Hex display/ 
modify; Mappable keys; “C” macro 
language; Dialog Definition 
Language; and much more ... 


. Intuitive and 
powerful, DevTrack provides an integrated 
client/server and intranet/Internet solution 
for defect management. Features universal 
ODBC support, extensive customization, 
powerful e-mail, and presentation-quality 
reports and graphs. 
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CodeWright 6.0 


by Premia 

New in CodeWright 6.0, the #1 selling 

programmer's editing system for the 

Windows platform: 

e CodeSense—word completion and 
function parameter help 

e CodeFolio—work faster by giving 
your teams a shared view of your 
most commonly used code 

e Sync Technology support for Visual Basic 
6.0 and Borland Delphi 4 

e Bi-directional synchronization for Visual 

Studio 97 and 98. 


FarPoint’s Input Pro 3 


by FarPoint Technologies 


Easily create professional data-entry 
screens using Input Pro's eight custom 
controls. Input Pro’s formatted-edit 
controls automatically validate 
date/time, numeric or text data 
entered by your user, database or as 
the application programmatically 
assigns or edits values. Use the 
boolean control to display a custom 
true/false/grayed value appearance. 
Use the memo control to display large 
amounts of text, even those greater 
than 64K. 
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by Blue Sky Software 


The Industry Standard in Help Authoring 
More software developers worldwide use best-selling RoboHELP Office to 

create full-featured Help and documentation systems for their applications. 
RoboHELP Office has won more awards than all other Help tools combined. 


@ Single source for Microsoft HTML Help, WinHelp, JavaHelp, WebHelp, 


Windows CE Help, and printed documentation 
e Easy to use WYSIWYG authoring environment 
e Quickly produce great online documentation 


e Deliver projects on time 


e Shorten the process of creating online Help 


e Generate printed manuals, training guides, 
e Y2K Compliant. 





by APEX Software Corporation 
Easily create stunning, lightening fast charts 
for Active Server Page (ASP) applications 
with NEW True WebChart 6.0, the ultimate 
charting tool for browser-independent Web 
server applications. Serve charts and graphs 
as JPEG or PNG to any browser! Or serve 
charts as an ActiveX control to allow your 
end-users to change chart types and data 
values! True WebChart 6.0 has built-in 
Olectra® Chart technology to provide chart 
types galore; data binding to any data source 
without writing a single line of code; zooming, 
scaling, and rotation; and much more. 





by SoftQuad Software, Inc. 


HoTMetaL PRO 6.0, is the world’s 
most versatile Website creation 
and management tool. Powerful, 
customizable and extensible, it 
gives professional developers the 
advanced capabilities and produc- 
tivity tools needed to quickly create 
sophisticated sites. With unrivaled 
editing, site management and 
publishing tools, HoTMetaL PRO 6.0 
is the ultimate Web development 
environment. 
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by SourceGear 


Access SourceSafe" 
over the Internet 
Connect to a centralized Visual 
SourceSafe database using any 
standard TCP/IP connection. 
IDE integration 

Over 10 times faster than RAS 
Efficient—uses data compression 
Secure—up to 128-bit data encryption 
Familiar—the look and feel of VSS 
Cross-Platform—98/NT/UNIX. 

tt Pro edition, 1 users. 
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by MicroEdge, Inc. 
Visual Slickedit®, the award-winning 
editor from MicroEdge, Inc. increases 
development productivity, and improves 
software quality. It supports most 
languages out of the box and is 
extendible to support your favorite 
language as well. 


With a multi-platform presence and 
integration with industry leading 
development environments, Visual 
SlickEdit provides an entire organization 
with a standard coding environment. 






by InstallShield 
Software Corp. 


InstallShield® Professional is the 
commercial developer's choice for 
creating industry-standard Microsoft® 
Windows® application installations. 
Critically acclaimed and used by over 
ninety of the top 100 independent 
software vendors, InstallShield 
Professional offers both scripting power 
and timesaving visual tools in an 
integrated development environment. 
The choice for intricate application 


by LEAD Technologies, Inc. 
LEADTOOLS is a family of 
comprehensive toolkits designed to 
help programmers integrate color, 
grayscale, document, medical, 
multimedia and vector imaging into 
their applications quickly and easily. 
Whatever your programming needs, 
LEAD has a toolkit specifically 
designed to give you the best 

imaging technology available with the 
stability and dependability you would 
expect from the imaging LEADer. 


requirements is InstallShield Professional. 
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by Peale 
Corporation 
Build scalable 
Windows 2000 
applications 
today 


The Microsoft® Visual Studio® 6.0 
development system now provides the 
comprehensive resources necessary for 
successful Windows® 2000 development. 
From building e-commerce Web 
solutions that take advantage of the new 
Windows 2000 clustering server to 
highly scalable data-driven business 
applications, Visual Studio 6.0 Enterprise 
Edition contains everything you need in 
one development suite. 








t Price after manufacturer's $100 mail-in rebate. 

















Secreta 





















es 
2 
- 
i 
. i 
: 
£ s 
= Hi Ao nreteetaees 
[i — 
a gig 
i we 
“= 
on Bsn 
F; — : eS pooner SS 
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The Dell Precision Workstation. 
For people who believe in possibilities. 


Customize Online with 


starting at FEATURES GRAPHICS CARD <> F-VALUE CODE: 89298-491028 
Up to dual Pentium® III Xeon™ Processors from Business Lease” starting at 64MB up to 2GB 100MHz ECC SDRAM 32MB Diamond Viper V770D (Upgradeable to: Appian Jeronimo Pro, 
500MHz—550MHz (RAID Capable) $97/Mo., 36 Months 9GB‘ up to 36GB‘ (10,000 RPM) Ultra-2/LVD Diamond Fire GL1 or Intense3D™ Wildcat™ 4000) 


SCSI Hard Drives 
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starting at: FEATURES GRAPHICS CARD “> FE-VALUE CODE: 89298-491021 
Up to dual Pentium III Processors from Business Lease” starting at 64MB up to 1GB 100MHz ECC SDRAM 32MB Diamond Viper V770D (Upgradeable to: Appian Jeronimo Pro, 
450MHz—600MHz (RAID Capable) $74/Mo., 36 Months 9GB‘ up to 36GB‘ (10,000 RPM) Ultra-2/LVD Diamond Fire GL1 or Intense3D Wildcat 4000) 

SCSI Hard Drives 

starting at FEATURES GRAPHICS CARD Z E-VALUE CODE: 89298-491019 
Up to dual Pentium III Processors from Business Lease” starting at 128MB up to 512MB PC800 RDRAM at 400MHz 32MB 4X AGP Diamond Viper V770D (Upgradeable to: Appian Jeronimo Pro) 
450BMHz-—600BMHz $67/Mo., 36 Months 6.4GB up to 27GB EIDE or 3GB*-36GB* (10,000 RPM) ~NEW 4X AGP in combination with new chipset and RDRAM 
NEW 133MHz System Bus increases data Ultra-2/LVD SCSI Hard Drives provides up to 25% improved intense graphics 


throughput/transfer rates for better 
system performance 
Please contact your sales representative for more information as estimated ship dates will exceed 30 days. 


starting at | FEATURES GRAPHICS CARD 4 Z E-VALUE CODE: 89298-491017 
Up to dual Pentium III Processors from Business Lease" starting at 64MB up to 512MB 100MHz ECC SDRAM 32MB Diamond Viper V770D (Upgradeable to: Appian Jeronimo Pro) 
450MHz—600MHz $60/Mo., 36 Months 6.4GB‘ up to 27GB* EIDE or 9GB‘—18GB* (10,000 RPM) 


Ultra-2/LVD SCSI Hard Drives 


COMMON FEATURES: 17° (16.0" vis) M780 Monitor [Upgrades Available from 17° (16.0" vis) P780 to 24" (22.5" vis) Multi-Monitor Capability], Microsoft® Windows NT°® 4.0, Integrated 3Com® 10/100 PCI NIC with Remote Wakeup, 40X Max° 
Variable EIDE CD-ROM Drive, Integrated Sound Blaster Pro Compatible Sound, Remote Client Manageability support via DMI 2.0, Wakeup On LAN Capable, 3-Year NBD On-site® Service, 7x24 Dedicated Hardware Tech Support 


AYA: Dell Precision Workstation 610 — PC Magazine's Editors’ Choice Award (5/99) and Digital Video's Award of Excellence (3/99) 
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Smallest Web Server 


A graduate student at the University of 
Massachusetts, H. Shrikumar, has devel- 
oped the most recent “world’s smallest 
web server’— literally smaller than a typ- 
ical fingernail. Researchers have been leap- 
frogging each other with ever-smaller 
web-server implementations, but this one 
will be hard to beat. The nearest com- 
petitor is a matchbox-sized 486SX imple- 
mentation from Stanford University’s Wear- 
able Computing Laboratory. While the 
Stanford web server is basically a full- 
fledged single-board PC costing several 
hundred dollars to manufacture (see 
http://wearables.stanford.edu/), Shriku- 
mar’s “iPic Server” is made from parts cost- 
ing a few dollars. 

The key to the iPic Server is that 
Shrikumar managed to shrink an RFC- 
1122-compliant TCP/IP stack to 256 bytes. 
Using an 8-bit PIC microcontroller cou- 
pled with a 32-KB EEPROM, the system 
can fit on a circuit board about the size 
of a match-head. A proof-of-concept 
demonstration (see http://www-ccs.cs 
-umass.edu/~shri/iPic.html) includes sev- 
eral Java applets and Postscript files de- 
scribing the system, all stored in 32 KB 
on the EEPROM. The tiny iPic Server 
would be ideal for controlling appliances 
and other simple devices via the Internet. 

Shrikumar told DDJ that “the biggest 
challenge was developing the TCP/IP 
stack. Setting up the system took only 
about 15 days, but it took over four 
months to develop a minimal TCP/IP 
stack that conformed to the standard and 
would deliver well-behaved packets.” In 
addition to the TCP/IP stack, Shrikumar 
wrote a tiny real-time operating system 
(essentially an RMA task-scheduler) and 
a simple telnet implementation for edit- 
ing files on the chip. 


CPU Flushed with Success 


It’s hard to imagine how a new micro- 
processor could have anything in com- 
mon with public restroom toilet partitions, 
but Advanced Micro Devices (AMD) has 
managed to make the connection. AMD’s 
new line of Athlon processors shares its 
name with a trademarked material called 
“Athlon,” manufactured by Trespa North 
America, Ltd. According to Trespa’s web 
site (http://www.trespa.com/), “Toilet Par- 
titions made from Trespa Athlon® panels 
function well in humid areas such as 
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gyms.” Lest there be any confusion, Athlon 
panels can be cleaned with soap and wa- 
ter, a practice not recommended for Athlon 
processors. The moral: Trademark search- 
es are a good thing. 


Light in the Slow Lane 


The principles of quantum mechanics 
are leading to advances in controlling 
light for optical applications. The quan- 
tum optics group at Texas A&M Univer- 
sity, led by physics professors Marlan 
Scully and George Welch, has come up 
with a way of slowing down light by a 
factor of 3 million using a relatively in- 
expensive and simple process. Heating 
the metal rubidium to 100 degrees Cel- 
sius, which turns it into a vapor, and then 
passing light through the heated gas, 
slows down the speed of light to about 
90 meters per second. The possibility of 
manipulating light at such slow speeds 
could lead to, among other things, new 
methods for controlling optical signals, 
storing optical data, and developing op- 
tical switches. 

The key to the process is that the light- 
absorptivity of the rubidium vapor can 
be controlled by applying a separate 
laser beam to the vapor. The laser beam 
changes the quantum state of the rubid- 
ium subatomic particles so that the va- 
por is virtually transparent, thus pre- 
serving the intensity of the light passing 
through the gas at the slow speed. And 
since the light intensity can be varied, 
the process can be applied to the con- 
cept of optical switching. Signal delay is 
another potential application for this pro- 
cess. The rubidium process can be 
achieved in a container the size of a test 
tube, potentially eliminating the need for 
extensive fiber optic cabling to produce sig- 
nal delays. For more information, see http:// 
focus.aps.org/v3/st37.html. 


Windows CE Gives You Gas 


Before long, pumping gas will join the 
ranks of just another online experience. 
BP Amoco, Microsoft, and gas-pump 
manufacturer Tokheim have teamed up 
to develop gas pumps running Windows 
CE with a web-browser terminal, from 
which you can check e-mail and surf the 
Internet while pumping gas. Whether 
this is a good idea remains to be seen. 
Still, it may be a boon for companies 
wanting to advertise their wares. Ac- 
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17, 1999), the average gas pump serves 
about 70 people per day. Multiply that 
by 100,000 pumps and an ad on a pump 
web terminal could reach 7 million con- 
sumers. 


The ACM Says No to Software 
Engineering Licensing 

The Association for Computing Machin- 
ery (ACM) has declined to support a plan 
by the Texas Professional Engineers Li- 
censing Board (http://www.main.org/ 
peboard/) to administer a licensing exam 
for software engineers, as it does for tra- 
ditional engineering disciplines such as 
mechanical and civil engineering. In a pol- 
icy statement, the ACM announced that it 
“is opposed to the licensing of software 
engineers at this time because ACM be- 
lieves that it is premature and would not 
be effective at addressing the problems of 
software quality and reliability.” Accord- 
ing to the policy statement, the ACM is 
committed to improving software quality 
by promoting research and development, 
and “identifying standards of practice.” 
Meanwhile, the Texas licensing board has 
postponed the establishment of a software 
engineering license. 


NT versus CE 


Muddying the OS waters as only it can 
do, Microsoft has announced Windows 
NT Embedded 4.0— a version of NT tar- 
geting OEMs developing high-end em- 
bedded products such as integrated 
voice response systems, PBXs, call cen- 
ter systems, medical appliances, and ad- 
vanced printers. The primary difference 
between standard NT 4.0 and NT Em- 
bedded 4.0, is that the embedded ver- 
sion has been “componentized,” letting 
you pick and choose services and pro- 
tocol stacks as needed. 

NT Embedded can run headless, that 
is, Without a graphical user interface, 
and on “spindleless storage,” such as 
flash memory, CD-ROM, and disk-on-chip 
systems. It can run on 486SX or more 
powerful processors with a minimum of 
8 MB of RAM and 12 MB of ROM. Vince 
Mendillo, a Microsoft product manager, 
told DD] that NT Embedded does not 
compete with Windows CE, which tar- 
gets lower-end embedded applications 
such as cell-phones, PDAs, and other 
devices running with as little as 40 KB 
of RAM and 256 KB of storage. 
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Jack J. Woehr 


he Mars Pathfinder Mission was the first mission of NASA’s 
Mars Surveyor program— a decade-long program of robot- 
ic exploration that focused on the search for evidence of 
past life on Mars, understanding the Martian climate and its 
lessons for the past and future of Earth’s climate, and under- 
standing the geology and resources that could be used to sup- 
port future human missions to Mars. As such, the mission (man- 
aged by the Jet Propulsion Laboratory; http://www .jpl.nasa.gov/) 
was one of the most ambitious and closely watched space mis- 
sions in history. 
Pathfinder essentially consisted of a stationary lander (the 
“Lander”) and surface rover (called “Sojourner”), which togeth- 
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er had the primary objective of demonstrating the feasibility of 
low-cost landings on and exploration of the Martian surface. 
The Pathfinder system itself was built around a variety of off- 
the-shelf hardware and software components. At the heart of 
the system was a single CPU— the RS6000— running the Wind 
River Systems VxWorks real-time operating system. 

Pathfinder was launched on December 4, 1996, and had a 
seven-month cruise to Mars. It landed on Mars on July 4, 
1997, where Sojourner almost immediately began conducting 
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experiments and collecting data; see Figure 1. In the first month 
of surface operations the mission returned to NASA about 1.2 gi- 
gabits of data, including 9669 Lander and 384 Rover images and 
about 4 million temperature, pressure, and wind measurements. 

Suddenly, on September 27, 1997, communication with So- 
journer was lost as the system began experiencing total resets, 
along with the resulting loss of data. The mainstream press 
promptly— and generally inaccurately — pounced on the prob- 
lem, referring to “software glitches” that were due to the com- 
puter “trying to do too many things at once.” 

The problem, as JPL engineers like Glenn Reeves (the “Flight 
Software Cognizant Engineer for the Attitude and Information 
Management Subsystem, Mars Pathfinder Mission”), quickly dis- 
covered involved priority inversion— a phenomenon familiar to 
real-time operating-systems engineers for more than 20 years. 
Contrary to most published reports, the problem was hardly ex- 
traterrestrial: A low-priority task (such as one for meteorologi- 
cal data gathering) occasionally grabbed a semaphore needed 
by a top-priority task (a bus management task, for example), 
then got preempted by medium-priority tasks (for instance, a 
communications task). Therefore, the high-priority task was not 
able to complete its work by the specified time. When the 
blocked top- priority task didn’t do its work in time, a watchdog 
thread reset the system. This reset reinitialized all of the hard- 
ware and software. It also terminated the execution of the cur- 
rent ground commanded activities. Data that had already been 
collected wasn’t lost. However, the remainder of the activities 
for that day could not be accomplished until the next day. 

Not only were JPL engineers aware of priority inversion from 
the outset, but they had carefully planned for instances of it. Con- 
sequently, they were able to quickly reproduce the problem on a 
duplicate system in the lab, then transfer a fix— changing the sys- 
tem default priority for select() semaphore creation— to the space- 
craft about 100 million miles away. (Talk about remote debug- 
ging!) This was doable because they had originally created two 
writable images of the control software, and either one could be 
deleted and reloaded from Earth. The team chose to never delete 
one of the two original copies while patching from the ground, 
for fear of losing something clean to boot with. 

In the end, all went well. The spacecraft completed its mis- 
sion. Reeves, who is now the Chief Engineer, Mission Data Sys- 
tem Project, recently took time to chat with DDJ/ contributing 
editor Jack Woehr about Pathfinder and other issues involved 
in writing software for extraterrestrial exploration. 


DDJ: You're programming computers that operate on the sur- 
face of another planet. That’s weird! 

GR: It’s weird for me, let me tell you. You’re in this microcosm 
of people where your whole life is concentrated on making sure 
the next day’s work of activities are going to go just right. You’re 
walking home at 3:00am still wired from staying up all night 
looking at information coming back. Your wife and the kids are 





Figure 1: Sojourner on the surface of Mars. (Courtesy 
NASA//PL/Caltech.) 
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asleep, there’s no way you can sleep and you've go to be back 
at 8:00AM anyway. So you turn on CNN and there’s your smil- 
ing face in the control room and you're looking at the same im- 
ages you looked at 12 hours ago. It’s a strange feeling. 

It was a small project by a small team, yet it had a phenom- 
enal impact overwhelming to many of us. On the technical side, 
it seemed straightforward at the time. 


DDjJ: | guess when you're doing the code, it’s like any other 
real-time project. 

GR: When you have a ballistic entry, there’s not much you can 
do about it. There’s a littke unknown, where you're relying on 
information gathered 22 years ago, but in general, we tried to 
simplify that job as much as we could. 

We spun the spacecraft, we had inertia on our side...the con- 
trol part was fairly straightforward. The hard part was thinking 
what things could go wrong, what do we do if something 
breaks...There are very few things that are redundant. How are 
we to go about testing it? How do we make sure that we can 
build an environment that matches what we want the spacecraft 
to fly through? Things like that turn out to be much harder than 
the spacecraft development itself. 


DDjJ: That's always a problem when one designs a large sys- 
tem, how one tests it in anything resembling the deployment 
environment. 

GR: Space is pretty forgiving, few forces, everything action and 
reaction. Coming down through the atmosphere, that’s a little 
different ballgame, but we can still do decent simulations. We 
gave up on trying to model bouncing and orientation of the ve- 
hicle and the simulation of the accelerometer values during the 
movements. Those were too hard to do. We tested the thing as 
hard as we can in a brute force sort of way. 


DDJ: You dropped it from a tall building? 
GR: Some drop testing. We did a lot of testing of the airbags. We 
proved to ourselves they could withstand the forces of the bounces, 
made sure they wouldn’t bottom out, would retain the right pres- 
sure, that the airbag material itself wouldn’t rip prematurely or 
catastrophically. We did that for two or three years, it was either 
the most expensive portion of our test program or pretty close. 

The actual orientation, opening the petals and turning the ve- 
hicle over, was very much brute force. Put the thing on the 
ground, read the accelerometers, see that it opened the right 
petal in the right order. 

We did testing of the airbags as they are retracted. They ge 
pulled in by cables within the bag. We did a lot of testing with 
rock placement, “Given the fact that bag’s going to wrap around 
this rock, are the motors going to stall and leave the bag ex- 
tended and flapping in the breeze?” 

In the end, the darn thing bounced 16 times, landed right on its 
base, and all the airbags retracted right in, just like it was supposed 
to. The perfect scenario happened in the actual mission. 


DDjJ: It seems almost petty to ask about the computer pro- 
gramming problem in comparison with the flow of the entire 
enterprise. 

GR: In this mission, most of the navigation was done from the 
ground. The attitude of the spacecraft was controlled onboard. 

JPL used to build their own computers; in fact, at a point in 
time, we built our own CPUs. Not any longer, that’s an art form 
that’s better left to industry. Single-board computers are an off- 
the-shelf item. 

So for the second time in history, JPL went out to buy a flight 
computer on contract. We put in a provision that specified an 
operating system on top of that. I wrote a lot of the flight com- 
puter specifications. 
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I was very familiar with [Mentor Graphics] VRTX and [Wind 
River Systems] VxWorks, and have done some [Integrated Sys- 
tems] pSOS work before. So I specified a fairly generic real-time 
operating system. What was proposed by IBM, the winner of 
the flight computer contract, was OSOpen, done by IBM’s Raleigh, 
North Carolina division. It was in early beta. 

We looked at it, and they had some more work to do on it. 
Then suddenly around 1993-1994, IBM Federal Systems got 
sold to Loral. What had been one company became two di- 
visions of two large companies that were mutually antago- 
nistic. It became obvious that we weren’t going to get a real- 
time operating system from these guys in Raleigh within the 
time frame we needed it. 

That’s when we decided to go to Wind River and see if they 
would come in and port for us to this RAD6000 computer. 


DDJ: The original IBM RISC mask, rad-hard? 

GR: Right. In a nutshell, we sold Wind River on the neat 
publicity they would get porting VxWorks onto this proces- 
sor. They bit, and did it fairly inexpensively and they did it 
quickly. We had a working version of VxWorks within four 
months. 

Not only were they receptive to the idea, but they put peo- 
ple on it right away. They were very generous with time, there 
was no nickel-and-diming on this contract. When there were 
problems...I had a team of seven people doing the flight soft- 
ware for this (the rover was entirely separate)...the technical 
interchange was engineer-to-engineer, there was no contract 
management in the middle. It was a nice interface. They did 
a good job. 


DDjJ: And when you had to hot-patch in flight? 
GR: That’s standard procedure. You always build in the ability 
to change it. 


DDJ: Just in case. 

GR: Just in case, but JPL and a bunch of decent-sized compa- 
nies have had the problem where you can’t get all the software 
done in time for launch. You always make sure you build the 
capability to change things. 


DDJ: When you send something 300,000,000 miles there’s al- 
ways the chance something you hadn’t anticipated will happen. 
GR: Well, as we said, space is forgiving, but on the surface 
of Mars and other places, you’re right. But what we'd really 
like to do is build spacecraft that are much smarter about how 
to take care of things they don’t precisely expect yet still 
achieve the things they are intended to do without interaction 
from the ground. 


DDjJ: Real robots! Were you an Asimov fan when you were a kid? 
GR: Still am! Both a kid and an Asimov fan. 


DDJ: What did your title on the Pathfinder mission —‘Flight 
Software Cognizant Engineer”’— signify? 

GR: I was ultimately responsible for development and operation 
of flight software on the spacecraft itself. My butt on the line. 


DDjJ: Software project management, the final frontier! How do 
you guys actually get the product out? 

GR: Here’s my opinion on why Pathfinder worked as well as 
it did: We were focused. We had a specific goal, to get this 
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Glenn E. Reeves 


he Mars Pathfinder spacecraft had a 
| RS6000-based single-board com- 
puter residing on a VME bus to con- 
trol the spacecraft. The single VME chas- 
sis also contained interface cards for the 
radio, camera, and an interface to a 1553 
bus. The 1553 bus, in turn, connected to 
the “cruise stage” and the “lander” part of 
the spacecraft. The hardware on the cruise 
part of the spacecraft controlled the 
thrusters, valves, sun sensor, and star scan- 
ner. The hardware on the Lander provid- 
ed an interface to accelerometers, radar 
altimeter, and an instrument for meteoro- 
logical science (ASI/MET). The hardware 
that was used to interface to the 1553 bus 
(at both ends) was inherited from the 
Cassini spacecraft. 
To support the Mars Pathfinder Mis- 
sion, Wind River Systems ported its stan- 
dard VxWorks for the 680x0 to the 
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RS6000. The RS6000 is the same single- 
chip CPU that can be found in some 
(now older) IBM AIX workstations. The 
Mars Pathfinder flight software also had 
several debug features that were used in 
the lab, but not on the actual flight space- 
craft because they produced more in- 
formation than we could send back to 
Earth. These features were not enabled, 
but remained in the software by design. 

One of these tools was a trace/log fa- 
cility that was originally developed to find 
a bug in an early version of the Wind Riv- 
er Systems VxWorks port. David Cum- 
mings, one of the JPL software engineers, 
built the trace/log facility. Lisa Stanley (at 
Wind River Systems) took this facility and 
instrumented the pipe services, msgQ ser- 
vices, interrupt handling, select services, 
and the tExec task. The facility initialized 
at startup and continued to collect data (in 
ring buffers) until told to stop. The facili- 
ty produced a voluminous dump of in- 
formation when asked. 

When the “repeated reset problem” 
(for details, see RISKS Digest, Volume 19 
Issue 54, January 10, 1998; http://catless 
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ncl.ac.uk/Risks/19.54.html) occurred on 
Mars, we ran the same set of spacecraft 
activities over and over again in the lab. 
Since the flight software had the trace/log 
facility enabled and the failing task was 
already coded so as to stop the trace/log 
collection and dump the data (even 
though we knew we could not get the 
dump in flight) for this error, we went 
into the lab to test whether we would 
have to change the software. 

In less than 18 hours, we were able to 
repeat the problem, isolate it to an inter- 
action of the pipe() and select() mecha- 
nisms, diagnose it as a priority inversion 
problem, and identify the most likely fix. 
In fact, the fix seemed straightforward: We 
had to change the creation flags for the 
semaphore used within the select() facil- 
ity so as to enable priority inheritance. This 
change was possible because Wind River 
Systems supplied global variables for pa- 
rameters, such as the “options” parameter 
for the semMCreate used by the select ser- 
vice (although this was not documented 
and those who do not have VxWorks 

(continued on page 26) 
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spacecraft launched, get it to Mars, get it through the atmo- 
sphere, deliver the rover onto the surface. We managed to fo- 
cus everyone on the team on that objective. 

Number two, JPL was in this Total Quality Management phase, 
so they talked a lot about empowerment, authority, and respon- 
sibility. Some of us, despite being more than a little skeptical about 
TOM itself, took that and ran with it. If we were empowered, that 
meant we really had the ability to make the decisions. The whole 
project worked that way, and the management of the project struc- 
tured it. They really did trust the people working on it. You really 
did take responsibility for the people you put on the project. 

We were the small project at JPL. Still ongoing was the back 
end of all the Cassini (see Figure 2) development, a 3000-person 
development. Pathfinder was 300 persons or fewer. So, the team 
was very tightly knit. 

All these things contributed to the success from the manage- 
ment point of view. I’d love to say that JPL took that lesson and 
applied it to all the subsequent projects, however, I would say 
the exact opposite was true. A lot of people look at Pathfinder 
as successful, yet, “We don’t want to do it that way again.” 

Recognize, however, that one of the reasons that’s true is that, 
in the past, we at JPL had these huge projects and were able to 
build up a lot of infrastructure institution-wide within a project. 
But since we have a bunch of much tinier projects now, we have 
to build an infrastructure across the board, so there’s a lot more 
interdependency between projects than there has been in the past. 


So the Pathfinder approach of putting yourselves out, a sort 
of skunkworks type of environment, just doesn’t work any- 
more at JPL. 


DDjJ: As space travel becomes routine, you don’t need genius- 
es to do it. 
GR: When that occurs! 


DDj: But that’s your job, year after year, trying to make it routine. 
GR: That’s correct. And once we make it routine, we’d like to 
make sure that industry is doing the routine part and we go off 
to the stuff that’s still on the frontier. 


DDJ: Glenn, what are you doing now? 
GR: I’m working on one of those infrastructure things. JPL is 
developing some avionics hardware that’s radiation hardened to 
the point it can survive at Europa, which has a very severe ra- 
diation environment. We are pushing the technology in this are- 
na, looking at the part level at one megarad type of problems. 
This is a lot of contractual work going outside of JPL. We’re 
trying to fly as much commercial heritage stuff as we possibly 
can. The flight computer we've currently selected is PowerPC 
750 based at 200 MHz. 


DDJ: You're putting a Macintosh into space! 
GR: Isn’t that convenient? The processor speed of the thing we 
flew on Pathfinder was about 22 MHz. We’ve gone up an order 


(continued from page 24) 

source code or have not studied the source 
code might be unaware of this feature). 
Still, the fix was not that straightforward 
for several reasons: 


1. The code for this was in the selectLibC) 
and was common for all device cre- 
ations. When this global variable is 
changed, all of the select semaphores 
created after that point will be creat- 
ed with the new options. There was 
no easy way in our initialization log- 
ic to only modify the semaphore as- 
sociated with the problem. 

. If we made this change and applied 
it on a global basis, we didn’t know 
how it would affect the behavior of 
the rest of the system. 

. Because Wind River Systems deliber- 
ately left the priority inversion option 
out of the default selectLib() service 
for optimum performance, we didn’t 
know if performance would degrade 
if we turned the priority inversion on. 

. Finally, we didn’t know if there was 
some intrinsic behavior of the select 
mechanism itself that would change if 
the priority inversion was enabled. 


In the end, we modified the global 
variable to include the priority inversion. 
This corrected the problem. We asked 
the Wind River Systems engineers to an- 
alyze the potential impacts for (3) and 
(4) above. They concluded that the per- 
formance impact would be minimal and 
that the behavior of select() would not 
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change so long as there was always only 
one task waiting for any particular file 
descriptor (this was true in our system). 
I believe that the debate at Wind River 
Systems still continues over whether the 
priority inversion option should be on 
as the default. As for the aforementioned 
(1) and (2), the change did alter the char- 
acteristics of all of the select semaphores. 
We concluded, both by analysis and test, 
that there was no adverse behavior. We 
tested the system extensively before we 
changed the software on the spacecraft. 

We weren't able to catch the problem 
before launch because the problem would 
only manifest itself when ASI/MET data 
was being collected and intermediate tasks 
were heavily loaded. Our before-launch 
testing was limited to the “best case” high 
data rates and science activities. The fact 
that data rates from the surface were high- 
er than anticipated and the amount of sci- 
ence activities proportionally greater 
served to aggravate the problem. We did 
not expect nor test the “better than we 
could have ever imagined” case. 

We did see the problem before land- 
ing, but could not get it to repeat when 
we tried to track it down. It was not for- 
gotten nor was it deemed unimportant. 
Yes, we were concentrating heavily on 
the entry and landing software. Yes, we 
considered this problem lower priority. 
Yes, we would have liked to have ev- 
erything perfect before landing. Howev- 
er, I didn’t see any problem, other than 
that we ran out of time to get the lower 
priority issues resolved. 
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We did have one other thing on our 
side —we knew how robust our system 
was because that is the way we designed 
it. We knew that if this problem occurred, 
we would reset. We built in mechanisms 
to recover the current activity so that there 
would be no interruptions in the science 
data (although this wasn’t used until later 
in the landed mission). We built in the abil- 
ity (and tested it) to go through multiple 
resets while we were going through the 
Martian atmosphere. We designed the soft- 
ware to recover from radiation induced 
errors in the memory or the processor. The 
spacecraft would have even done a 60- 
day mission on its own, including de- 
ploying the rover, if the radio receiver had 
broken when we landed. There were a 
large number of safeguards in the system 
to ensure robust, continued operation in 
the event of a failure of this type. These 
safeguards allowed us to designate prob- 
lems of this nature as lower priority. We 
had our priorities right. 

Did we (the JPL team) make an error 
in assuming how the select/pipe mecha- 
nism would work? Probably. But there was 
no conscious decision to not have the pri- 
ority inversion enabled. We just missed it. 
There were several other places in the 
flight software where similar protection 
was required for critical data structures 
and the semaphores did have priority in- 
version protection. A good lesson when 
you fly commercial off-the-shelf stuff— 
make sure you know how it works. 


DDJ 
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of magnitude. That’s a phenomenal increase. JPL has, historically, 
flown processor technologies that are 10 to 12 years behind... 


DDJ:...As you wait for the rad-hard version of the part. 

GR: Yup. What I’m specifically working on is a project that 
is sort of mislabelled as “Mission Data System.” It’s really an 
institution-wide effort to do a couple things. JPL has had sev- 
eral autonomy- oriented things occurring. We’re trying to come 
up with an architecture that builds a foundation where we 
can have much more autonomous spacecraft than we’ve had 
in the past. 

There are a lot of things involved with that, not just software, 
but also process, normal, up-to-date software engineering prac- 
tices. I think JPL is moving from a hardware-centric organiza- 
tion to a more software-centric organization. The computer sys- 
tems that we’re building now look a lot like little local area 
networks. Computer systems both fast and less fast connected 
by things like FireWire and I2C. 


DDjJ: 12C is still going strong! 

GR: It’s not fast, but it’s low power. That’s often a real determi- 
nant. For high volume, high speed, we have 1394 FireWire — 
100 MBit/sec. 


DDjJ: | guess the NASA environment is pretty “toy-rich” for the 
programmer? 
GR: We're trying to make it toy-rich in the sense that we’re push- 
ing hard for the commercial, off-the-shelf standardized things. 
On Pathfinder we said, “We’re going to fly the VME bus, be- 
cause look at all the cards we can buy instead of doing in-house 
design and building every piece of test equipment.” 
We’re still following that path. 


DDJ: We started discussing the challenge of your current as- 
signment in the context of trying to preserve the best of the 
Pathfinder skunkworks methodology in the current horizontal 
arrangement of smaller teams and standardized parts. How are 
you addressing that challenge? 

GR: JPL is in the throes of trying to address team organization 
and interdependency between teams effectively. It’s a big prob- 
lem for an organization to go from a very command and control- 
oriented to a very interdependency- oriented structure. 


DDj: Is it left to JPL internally, or are you one eddy in the 
NASA river? 

GR: At this point it’s pretty much JPL internal. Our interdepen- 
dency issues are certainly known to NASA. 


Figure 2: Computer generated image of the Cassini 
spacecraft. (Courtesy NASA/JPI/Caltech.) 
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Eee sammmmmmmmmnenemmemanll 


DDjJ: You personally are being slurped into management. Do 
you get to write code anymore these days? 

GR: Not these days...I have a position called Mission Data Sys- 
tem Chief Engineer. It sounds like a technical decision making 
position, but it’s primarily a programmatic activity. I’m resisting 
as best I can! 


DDjJ: All the techies reading DDJ are cheering, “Don’t give in 
to the Dark Side!” 

GR: Someone has to put the plan together. That deficiency has 
dragged me to the dark side. 


DDjJ: Yes, really, the management is the final frontier. Develop- 
ment environments are shrink-wrap now. Figuring out how to 
divide work suitably between people, how to interleave so the 
product arrives on time... 

GR: Those are second-order effects. The first-order effects are 
methodology. UML or not UML? Languages, classic things, the 
ability to do things on the type of computers we’re now buy- 
ing...Some of the things we had to worry about in the past we 
don’t have to worry about any more. A 200-MHz PowerPC is 
about 200 times faster than the processors that run in Cassini. 
A lot of efficiency issues are starting to disappear. 


DDJ: So you just write “good old code” like anybody writes? 
GR: Well, wouldn’t that be nice? Wouldn’t it be nice if we could 
get close to the maturing edge of some software engineering? 
JPL, because we've been relegated to targets about 10 to 15 years 
back, we ended up having to use whatever's available, but we're 
no longer in that ballgame. 


DDJ: What about Linux? 

GR: We haven’t evaluated it from a flight perspective. It has a 
fairly decent presence here on desktop computers. Every engi- 
neer has his opinion on the One True Way. Languages turn out 
to be a religious issue, too. 


DDJ: What religions are sweeping JPL? 
GR: Most of the work JPL has done has been primarily in pro- 
cedural languages. We did a fair amount of Ada on Cassini, but 
from a true object-oriented perspective, we haven’t been there 
yet, not on the flight side. Pathfinder and others are all C-based. 
But moving towards an object-oriented direction takes you to 
an object-oriented language, C++ or Java...We have a whole 
bunch of artificial intelligence folks for whom LISP is a language 
of choice. Factor in a multitasking, preemptive real-time envi- 
ronment, even a multilanguage environment, and you have a 
whole bunch of religious conflicts. 


DDjJ: You're going to need memory to run that stuff. 

GR: Look at the memory densities we’re starting to see. On 
Pathfinder we had 128 megabytes of RAM. We moved into the 
realm where my desktop machine has more than the space- 
craft. We’re continuing in that direction. Let’s put it this way, 
the software guys will be able to use all of that space. “If it’s 
there, we can fill it up.” 


DDJ: When are we going to Europa? 
GR: The hardware and the software come together in the two- 
year time frame for the two missions that will be supported. The 
first one is a mission called “Space Technology 4,” or ST4. It’s 
going off to a comet [to] take some samples [and] possibly even- 
tually return. I’m not sure about the return part now, they’ve 
done some descoping lately. 

The second mission is to Europa. They’re launching possibly in 
2003. There are other ongoing missions that will also probably 
use this technology, but those are the only two signed up now. 
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Glenn Reeves 


DDJ: What keeps people going to Congress for funding to shoot 
tin cans at Europa? Why does this happen? 

GR: My personal opinion is that you have to recognize this is a 
very small planet. At some point in the future, man is going to 
move off into space, there’s no doubt about it. People recog- 
nize there has to be some level of exploration that’s ongoing to 
support that goal. 

Perhaps the question is, “How can we spend billions in space 
while there are starving children in the world?” I think it’s a bal- 
ance, where you're balancing the short-term goals with the long- 
term goals. 

I can’t speak for the manned exploration side of NASA, but 
we on the robotic exploration side have witnessed great changes, 
going from billion-dollar spacecraft to $200 million spacecraft. 
NASA applies fairly small resources to greater and greater dis- 
covery and exploration missions. 


DDj: 1 grew up with Robert Heinlein’s books about mining the 
asteroids. Is that in the foreseeable future? 

GR: That’s interesting, because this week or next week there’s 
a guy coming from a company called SpaceDev Inc., to talk to 
us about commercial deep-space exploration. How there will 
be a shift from government-sponsored space exploration, whether 
Russian, Japanese, German, French, or American. I’m not sure 
he’s going to address mining, but he’s going to address that 
there are institutions both research and commercial that are will- 
ing to pay for images and spectroscopy in the hopes of dis- 
covering minerals, resources... 


DDj: The classic SciFi goals of space exploration. 
GR: I can tell you that it’s coming. Any time soon? Maybe in our 
lifetime, late...My kids will see it. 


DDjJ: Any advice for software engineers interested in space? 

GR: My brother-in-law says, “Ya write the software but ya 
never actually know about the thing it’s doing.” I think that’s 
wrong. I think you can still be a computer scientist and do phe- 
nomenally interesting applications of that art. Pathfinder is an 
example of how exciting that can really get. 


DDJ 
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FIFO versus 
shared memory 





Frederick M. Proctor 


hen dealing with real-time sys- 
tems, the overhead of interpro- 
cess communications (IPC) be- 
comes important. The formalized 
structures that are used in Linux for IPC 
can carry with them a significant amount 
of overhead. This can create timing prob- 
lems for your applications. In this article, 
I'll examine two of the best IPC mecha- 
nisms available under Linux—FIFOs and 
shared memory. 

Communication between Linux and Real- 
Time Linux (RT-Linux) processes is usually 
accomplished via first-in-first-out (FIFO) 
connections — point-to-point queues of serial 
data analogous to UNIX character devices. 
FIFOs have the following characteristics: 





e FIFOs queue data, so no protocol is re- 
quired to prevent data overwrites. 

e Boundaries between successive data 
writes are not maintained. 

e Applications must detect boundaries be- 
tween data, particularly if data is of vary- 
ing size. 

e FIFOs support blocking for synchro- 
nization. Processes need not poll for the 
arrival of data. 

e FIFOs are a point-to-point communica- 
tion channel. FIFOs do not support the 
one writer/several readers model. 


Fred is a member of the Intelligent Systems 


Division at NIST. He can be contacted at 
frederick. proctor@nist.gov. 
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Linux, 





& IPC 


You declare the maximum number of 
FIFOs in rt_fifo_new.c as #define RTF_NO 
64. These appear as devices dev/rtf0..63 
in the filesystem. This limit can be changed 
and the rt_fifo_new.o module recompiled. 
There is no limit to the number of FIFOs 
an application can use, or to the size of 
data that can be written to a FIFO (with- 
in practical memory limits). 

An alternative to FIFOs is shared memo- 
ry, in which a portion of physical memory 
is set aside for sharing between Linux and 
real-time processes. In a nutshell, shared 





memory is a pool of memory segments re- 
served at boot time. These segments may 
be mapped into the address space of more 
than one application, allowing for fast data 
sharing, data updates, and handshaking. This 
allows for low-latency parallel updates. 

Shared memory has the following char- 
acteristics: 


e It does not queue data written to it. Ap- 
plications requiring handshaking must 
define a protocol to assure data is not 
overwritten. 


Dr. Dobb’s Journal, November 1999 


Real-Time Linux, 


¢ Because data is not queued, individual 
items in data structures (megabytes in 
size) can be quickly updated. 

e It has no point-to-point restriction. 
Shared memory can be written or read 
by any number of Linux or real-time 
processes. 

¢ The number of independent snared 
memory channels is limited only by the 
size of physical memory. 

¢ Blocking for synchronization is not di- 
rectly supported. To determine if data 
is new, the data must contain a count 
that can be compared against previ- 
ous reads. 

e Mutual exclusion of Linux and RT-Linux 
processes is not guaranteed. Interrupt- 
ed reads and writes can be detected, 
however. 


Whether you use FIFOs or shared 
memory depends on the application’s 
natural communication model. For con- 
trol applications involving processes that 
execute cyclically based on the expira- 
tion of an interval timer (where data 


_ queuing is the exception rather than the 


rule), shared memory is a good choice 
for communication. 


Setting up the Shared-Memory Pool 
The shared-memory pool is a block of 
physical memory set aside at boot time so 
that Linux does not use it for processes. 
To set up the pool, you first determine 
how much physical memory the system 
has and how much is to be used for 
shared memory. Subtracting the size of 
the shared memory desired from the size 
of physical memory gives the absolute 
base address of the pool. The result is 
passed to the Linux boot loader (LILO) at 
boot time. This is accomplished by edit- 
ing /etc/lilo.conf and inserting a line with 
the append keyword. 
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(continued from page 32) 

Suppose, for example, the system has 32 
MB of memory and 1 MB is to be used for 
the shared-memory pool. The base address 
for shared memory is 32 MB—1 MB=31 MB. 
Assuming the original /etc/lilo.conf file con- 
tained Example 1(a), the file should be 
modified like Example 1(b). 

Similarly, suppose the system has 16 
MB of memory and 512 KB are to be used 
for the shared-memory pool. The base ad- 
dress for shared memory is 16384 KB—512 
KB=15872 KB. The /etc/lilo.conf file 
should be modified like Example 1(c). 

The size of the shared-memory pool 
must be less than the page size declared 
in /usr/include/asm/param.h. In Intel 
Pentium-class machines and above, the 
page size is 4 MB; on earlier machines, 
the page size is 1 MB. 


Addressing the 

Shared-Memory Pool in C Programs 
The base address of the shared-memory 
pool needs to be declared in C so that both 
Linux and RT-Linux code can reference it. 
For example, you can access shared mem- 
ory based at 31 MB using the C statement 
#define BASE_ADDRESS (31 * Ox 100000). 
Similarly, shared memory based at 15872 
KB may be accessed using the C statement 
#define BASE_ADDRESS (15872 * 0x400). 
This address is used differently in Linux 
than in RT-Linux. Linux processes need to 
map this physical address into their virtu- 
al address space. RT-Linux processes can 
reference data located at this address as a 
pointer directly. 

In addition to this declaration, the Lin- 
ux and RT-Linux C code must agree on 
the data structures written into shared 
memory. In Listing One, for example, a 
Linux process sends a command to an RT- 
Linux process by filling in the MY_COM- 
MAND structure and writing it into shared 





Example 1: (a) Original /etc/lilo.conf 
file; (b) modified file for 31 MB; (c) 
modified file for 15872 KB. 
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memory. The RT-Linux process reads this 
structure from shared memory to get the 
command. An RT-Linux process sends sta- 
tus to a Linux process by filling in the 
MY_STATUS structure and writing it into 
shared memory. The Linux process reads 
this structure from shared memory to get 
the status. 

The MY_STRUCT structure is a combi- 
nation of both the command and status 
structure, and can be used to ensure that 
the two structures do not overlap and that 
their fields are aligned on the proper 
boundaries. It is also possible to define 
two base addresses— one for each struc- 
ture— making sure the start of one struc- 
ture is after the end of the previous one 
and that all fields are properly aligned. By 
combining them into a single aggregate 
structure and letting the compiler allocate 
storage, the structure will have a valid byte 
alignment automatically. 

In a typical application, the BASE_AD- 
DRESS declaration and shared-structure 
declarations would be put in a header file 
shared by both Linux and RT-Linux code. 


Accessing the Shared-Memory Pool 
from Processes other than RT-Linux 
Normal Linux processes are required to 
map physical memory into their private 
address space to access it. To do this, the 
Linux processes calls open() on the mem- 
ory device /dev/mem; see Example 2. 

Due to security concerns, the default 
permissions on /dev/mem allow only root 
processes to read or write /dev/mem. To 
access physical memory, the program must 
be run as root, its permissions must be 
changed to setuid root, or the permissions 
on /dev/mem must be changed to allow 
access to users other than root. 

After the file descriptor is opened, the 
Linux process maps the shared memo- 
ry into its address space using mmap(); 
see Listing Two. BASE_ADDRESS is 
passed to mmap(), which returns a 
pointer to the shared memory as 
mapped into the Linux process’s address 
space. Once the shared memory is 
mapped, it may be accessed by deref- 
erencing the pointer, as in Example 3(a). 
When the process terminates, you use 
munmap() to unmap the shared mem- 
ory by passing the pointer and the size 
of its object; see Example 3(b). 





Example 2: Calling open() on the 
memory device /dev/mem. 
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Accessing the 

Shared-Memory Pool from RT-Linux 
Shared-memory access is much easier in 
RT-Linux since the real-time code exe- 
cutes in kernel space and thus is not re- 
quired to map physical addresses to vir- 
tual addresses. In Linux kernels 2.0.XX, 
the pointer can be set directly; see Ex- 
ample 4(a). In Linux kernels 2.1.XX, the 
pointer needs to be mapped via a call to 
the __va() macro defined in /usr/in- 
clude/asm/page.h, as in Example 4(b). 


Detecting New Writes 

FIFOs have an advantage over shared mem- 
ory in that reads and writes follow standard 
UNIX conventions. Processes other than 
RT-Linux processes can use write() to 
queue data onto a FIFO, and read() re- 
turns the number of characters read. Zero 
characters read from a FIFO means no new 
data was written since the last read. On the 
RT-Linux side, a handler is associated with 
a FIFO that is invoked after a nonreal-time 
Linux process writes to the FIFO. Normal- 
ly the handler calls 7tf_get() to dequeue 
the data from the FIFO. 

These functions are not necessary with 
shared memory. Reads and writes are ac- 
complished by reading and writing directly 
to pointers. Consequently, the operating 
system provides no way to detect if the 


contents of shared memory have been up- | 


dated. You need to set up this handshak- 
ing explicitly. 

One way to do this is to use message 
identifiers that are incremented for each 
new message. The receiver then polls the 
shared-memory buffer and compares the 
current identifier with the previous one. 
If they are different, a new message has 
been written. Handshaking to prevent mes- 
sage overrun is implemented by echoing 
message identifiers in the status structure 
once they have been received. New mes- 
sages are not sent until the status echoes 
the message identifier. 

This presumes a time-cyclic polling 
model on the part of both the Linux and 
RT-Linux processes. If this is not the nat- 
ural model for the application, then shared 
memory may not be a good choice for 
communication. If shared memory is re- 
quired for other reasons, and polling is 
not desirable, other synchronization be- 
tween Linux and RT-Linux processes can 
be used. For example, RT-Linux FIFOs 
can be used only for their synchroniza- 
tion properties. A byte written to a FIFO 
can be used to wake up a Linux process 
blocked on a read, or to call the handler 
in an RT process. 


Realizing Mutual Exclusion 

It is possible (and therefore a certainty) 
for a Linux process to be interrupted by 
a real-time process while in the middle of 
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(continued from page 34) 

a read/write to memory they are sharing. 
If the Linux process is interrupted during 
a read, the Linux process sees stale data 
at the beginning and fresh data at the end. 
If the Linux process is interrupted during 
a write, the real-time process sees fresh 
data at the beginning but stale data at the 
end. Both problems are fatal in general. 

The problem of ensuring consistency 
of data shared between two processes 
is the subject of operating-systems re- 
search, and general solutions exist (see, 
for instance, An Introduction to Oper- 
ating Systems, Second Edition, by Har- 
vey M. Dietel, Addison-Wesley, 1990). 
Our problem is simpler because only 
the Linux process can be interrupted. In 
no case can a Linux process interrupt a 
real-time process during the execution 
of its task code. 

This simplification means that an inuse 
flag can be used by the Linux process to 
signal that it is accessing the shared mem- 
ory. You declare the inuse flag at the be- 
ginning of the shared-memory structure, 
and it is set by the Linux process when it 
wants to read or write and cleared when 
the Linux process is finished. The real- 
time process checks the inuse flag before 
it accesses shared memory; if set, the pro- 
cess defers the read or write action until 
it detects that the flag is cleared. 

This may lead to the indefinite post- 
ponement of the real-time process, if the 
following conditions are true: 


e The real-time process runs at the peri- 
od of the Linux process, or at multiples 
of this period. 

e The accesses are synchronized so that 
the real-time process always interrupts 
the Linux process during the critical sec- 
tion when it has set the inuse flag. 


The first condition implies that the real- 
time code is running as slow or slower 
than the Linux code, and the second im- 
plies that the Linux code is running as de- 
terministically as the real-time code. Nei- 
ther is typically true, and both are rarely 





Example 3: (a) Dereferencing the 
pointer; (b) passing the pointer and 
the size of its object. 


36 


true at the same time. If these conditions 
are true for a system, successive deferrals 
can be detected by the real-time code and 
can trigger actions to keep the system un- 
der control. 

Listing Three illustrates the application 
of the inuse flag for commands written 
by the Linux process to the real-time pro- 
cess, and status written by the real-time 
process and read by the Linux process. 
Assume that command_ptr has been set 
up in a Linux process to point to the 
shared-memory area for commands to 
the real-time process. To write to shared 
memory, you must compose the com- 
mand, set the inuse flag in shared mem- 
ory directly, write the command, and re- 
set the inuse flag; see Listing Four. 

Assuming that the real-time code has 
set command_ptr to point to the shared 
memory for commands, it would read 
commands as in Listing Five. To read sta- 
tus information, the Linux process sets the 
inuse flag before copying out the data. 
Assuming that status_ptr has been set up 
in a Linux process to point to the shared- 
memory area for status from the real-time 
process, this would look like Listing Six. 

When writing status, the real-time pro- 
cess checks for the inuse flag and defers 
a status write if it is set. Assuming that the 
real-time code has set status_ptr to point 
to the shared memory for status, this 
would look like Listing Seven. 


Queuing Data in Shared 

Memory Using Ring Buffers 

While shared memory is most naturally 
suited for communications in which data 
overwrites the previous contents, queu- 
ing can be set up using ring buffers. Ring 
buffers queue 0 or more instances of a 
data structure, up to a predetermined 
maximum. 

To illustrate the use of ring buffers, con- 
sider a system that queues error messages. 
Errors are declared as strings of a fixed 
maximum length, and there is a fixed max- 
imum number of errors that can be 
queued. This is implemented as a two- 
dimensional array, as in Listing Eight. 





Example 4: (a) In Linux kernels 
2.0.XX, the pointer can be set directly; 
(b) in Linux kernels 2.1.XX, the 
pointer needs to be mapped via a call 
to the ___vaQ) macro. 
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Supplementing the actual list of errors 
are indices to the start and end of the 
queue, which wrap around from the end 
of the shared-memory area to the begin- 
ning (hence the name “ring buffer”), and 
a count of the errors queued. As previ- 
ously described, an inuse flag is also de- 
clared to signal that a Linux process is ac- 
cessing the ring buffer to prevent data 
inconsistencies in the event a real-time 
process interrupts Linux process access. 
The full shared-memory structure decla- 
ration is then Listing Nine. 

Both Linux and RT-Linux use the same 
access functions. However, Linux pro- 
cesses need to set the inuse flag before 
getting an error off the ring, and real-time 
processes need to check the inuse flag 
and defer access until the flag is zero. As- 
suming that errlog is a pointer to the 
shared-memory area for both Linux and 
real-time processes, the access functions 
look like Listing Ten. 

For Linux, getting an error off the ring 
buffer is accomplished by Listing Eleven. 
For an RT process, writing an error to the 
ring looks like Listing Twelve. 


Sample Code 

I've included a sample application that il- 
lustrates Linux to real-time commands, 
real time to Linux status, and real time to 
Linux error logging. 

The application consists of two parts. 
The first is a real-time process that runs 
cyclically, reads a command buffer, con- 
tinually updates a status buffer, and logs 
some diagnostic messages to the queued 
error buffer. The second part is a command- 
line Linux program that handles a few key- 
board commands for sending commands 
and printing the real-time process status 
and error log. 

This application is available electroni- 
cally; see “Resource Center,” page 5. The 
application is provided as a ZIP file and 
a gzipped tar file (shmex.tgz). Unpack and 
compile with: 


tar xzvf shmex.tgz 
make 


You have to set up Linux to boot with 
shared memory set aside, as detailed pre- 
viously. 
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Listing One 


typedef struct 
{ 


unsigned char inuse; /* more on this later */ 
int command; 
int command_number ; 
int arg1; 
int arg2; 
} MY_COMMAND ; 
typedef struct 
{ 


unsigned char inuse; /* more on this later */ 
int command_echo; 
int command_number_echo; 
int stati; 
int stat2; 
} MY_STATUS; 
typedef struct 
{ 


MY_COMMAND command; 
MY_STATUS status; 
} MY_STRUCT; 


e e 
Listing Two 
#include <stdlib.h> /* sizeof() */ 
#include <sys/mman.h> /* mmap(), PROT_READ, MAP_FILE */ 
#define MAP_FAILED ((void *) -1) /* omitted from Linux mman.h */ 
#include "myheader.h" 
MY_STRUCT *ptr; 
ptr = (MY_STRUCT *) mmap(@, sizeof (MY_STRUCT), 

PROT_READ | PROT_WRITE, MAP_FILE | MAP_SHARED, fd, BASE_ADDRESS) ; 
if (MAP_FAILED == ptr) 
{ 


/* handle error here */ 


} 
close(fd) ; /* £d no longer needed */ 


Listing Three 


typedef struct 
1 


unsigned char inuse; 
int command; 
int command_number; 
int argl; 
int arg2; 

} MY_COMMAND ; 

typedef struct 


unsigned char inuse; 

int command_echo; 

int command_number_echo; 
int stat1; 

int stat2; 

MY_STATUS; 


ond 


e ° 
Listing Four 
MY_COMMAND my_command; 

/* compose command in local structure */ 
my_command.inuse = 1; /* will overwrite during copy, so set here too */ 
my_command.command = 123; 
my_command.command_number++; 
my_command.arg1 = 2; 
my_command.arg2 = 3; 
/* set inuse flag */ 
command_ptr->inuse = 1; 
/* copy local structure to shared memory */ 
memcpy(command_ptr, &my_command, sizeof (MY_COMMAND) ) ; 
/* clear inuse flag */ 
command_ptr->inuse = 9; 


Listing Five 


if (@ != command_ptr->inuse) 


/* ignore it, perhaps incrementing a deferral count */ 
} 

else 

{ 


/* okay to access shared memory */ 


e e e 
Listing Six 
MY_STATUS my_status; 
/* set inuse flag */ 
status_ptr->inuse = 1; 
/* copy shared memory to local structure */ 
memcpy (&my_status, status_ptr, sizeof(MY_STATUS) ); 
/* clear inuse flag */ 
status_ptr->inuse = @; 
/* refer to local struct from now on */ 
if (my_status.stat1 == 1) 


atk 


e e 
Listing Seven 
if (0 != status_ptr->inuse) 
{ 

/* defer status write, perhaps incrementing deferral count */ 

} 
else 
{ 


/* okay to write status */ 
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e e e 
Listing Eight 
#define ERROR_NUM 64 /* max number of error strings to be queued */ 


#define ERROR_LEN 256 /* max string length for an error */ 
char error [ERROR_NUM] [ERROR_LEN] ; 


e e e 
Listing Nine 
#define ERROR_NUM 64 /* max number of error strings to be queued */ 


#define ERROR_LEN 256 /* max string length for an error */ 
typedef struct 
{ 


unsigned char inuse; 

/* flag signifying Linux accessing */ 
char error [ERROR_NUM] [ERROR_LEN]; /* the errors themselves */ 
int start; 

/* index of oldest error */ 


int end; 

/* index of newest error */ 
int num; 

/* number of items */ 

} MY_ERROR; 


e e 
Listing Ten 
/* initialize ring buffer; done once, perhaps in init_module() */ 
int error_init(MY_ERROR *errlog) 
{ 


errlog->inuse = @; 
errlog->start = @; 
errlog->end = @; 
errlog->num = @; 
return 9; 

} 


/* queue an error at the end */ 
int error_put(MY_ERROR *errlog, const char *error) 
{ 
if (errlog->num == ERROR_NUM) 
{ 


/* full */ 
return -1; 
} 
strncpy (errlog->error[errlog->end], error, ERROR_LEN) ; 
errlog->end = (errlog->end + 1) % ERROR_NUM; 
errlog->num++; 
return 9; 
} 
/* dequeue the error off the front */ 
int error_get(MY_ERROR *errlog, char *error) 
{ 
if (errlog->num == @) 


/* empty */ 
return -1; 
} 
strncpy(error, errlog->error[errlog->start] , ERROR_LEN) ; 
errlog->start = (errlog->start + 1) % ERROR_NUM; 
errlog->num--; 
return 9; 


} 


e e 
Listing Eleven 

char error[ERROR_LEN]; /* place to copy error */ 
/* set in-use flag in shared memory */ 
errlog->inuse = 1; 

/* copy error out */ 

if (@ != error_get(errlog, error)) 


/* empty */ 


else 
{ 
/* handle it */ 
} 
/* clear in-use flag in shared memory */ 
errlog->inuse = Q; 


Listing Twelve 


char error[ERROR_LEN]; /* place to compose error */ 
/* check for in-use */ 


if (®@ != errlog->inuse) 
{ 
/* defer writing, perhaps incrementing deferral count */ 
} 
else 
{ 


/* compose it */ 
strepy(error, "your error here"); 


if (@ != error_put(errlog, error)) 
{ 

/* full */ 
} 


} 
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Inside Windows NI 
System Data 








Exploring a 
mysterious function 





Sven B. Schreiber 


f you have ever written debugging soft- 

ware for Windows NT, you know it 

doesn’t take long before you can get 

stuck. Things as simple as enumerating 
processes or querying system performance 
are difficult due to the lack of detailed 
documentation. In an interesting Microsoft 
Systems Journal (MS) article series, Matt 
Pietrek presented two methods for getting 
at some internal system data. The first one 
involves querying the so-called Perfor- 
mance Counters stored in the registry (see 
“Under The Hood,” MS/, March/April, 
1996). The other method uses a special- 
purpose system module named psapi.dll 
(“Under The Hood,” MS/, August 1996). 
Both methods work well, but give just a 
tiny glimpse of the wealth of data used 
internally by Windows NT. 


NfQuerySystemInformation() 

While trying to figure out how the kill.exe 
utility (from the NT Resource Kit) works, 
I discovered it uses the mysterious Nf- 
QuerySystemInformation() function ex- 
ported by ntdll.dll to get a list of processes 
currently running. With this list, kill.exe is 
able to force termination of a process by 
name, simply by looking up the name in 
the process list and retrieving the associ- 
ated process ID, which is stored in this 
list, too. This looked interesting enough 
to warrant a closer look. To my surprise, 


Sven is a developer in Herzogenaurach, Ger- 
many, and be contacted at sbs@orgon.com. 
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that NtQuerySystemInformation() is ap- 
parently the general front door from user- 
mode code into internal kernel-mode data 
structures. 

One of the things that puzzled me most 
was the complete lack of documentation 
on this function. Even the NT Device Driv- 
er Kit (DDK) header files ntddk.h and nt- 
def.h, that usually provide answers to 
many questions about NT internals, are 
amazingly ignorant about any aspects of 
NtQuerySystemInformation(). No API 
prototype, no info class definitions — just 
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a single mention of the name in the com- 
ments to the definition of the structure 
CONFIGURATION_INFORMATION. 
Searching for “NtQuerySystemInformation” 
in the DDK help yields no hits at all. That’s 
about as undocumented as a system API 
ever could be! In this article, I'll reveal 
what kind of data this API gives you ac- 
cess to, and provide a DLL and demo pro- 
gram that lets you display most of that 
data on the screen (both available elec- 
tronically; see “Resource Center,” page 5). 

It’s simple to find out what parameters 
have to be passed to this function— you 


Dr. Dobb’s Journal, November 1999 








simply check how the NT core modules 
make use of it. Example 1 is the proto- 
type of NtQuerySystemInformation()), as 
well as that of its counterpart, NiSetSys- 
temInformation( ). Actually, they look sim- 
ilar to other (documented) NT info query 
APIs, such as NtQueryInformationPro- 
cess() or ZwQueryValueKey( ). All of them 
use an info class parameter, which is just 
an index number that selects the kind of 
information to be returned. pData points 
to a data buffer allocated by the caller, 
the size of which is indicated by dSize. 
On return, pdSize contains the number 
of bytes actually copied to the buffer. 
Like its friends, NtQuerySystemInforma- 
tion() returns an NTSTATUS value. If STA- 
TUS_INFO_LENGTH_MISMATCH 
(OxCOOO00004) is returned, the buffer was 
too small to hold the data. 


A Simple System Info Spy 

To find out more about the data supplied 
by NtQuerySystemInformation(), I wrote 
the ntqsi.exe utility (available electroni- 
cally), a simple console-mode application 
that outputs a hex dump of the data re- 
turned by NtQuerySystemInformation( ). 
The problem with this function is that it 
returns several types of information, de- 
pendent on the specified info class. I’ve 
observed the following variations: 


e A fixed-size data block (usually a scalar 
value or some structure). 

e A variable-size list of fixed-size items. 

e A variable-size list of variable-size items. 

e A list of lists. 


As you see, almost any kind of data you 
can think of is included. This makes it dif- 
ficult to write a general-purpose spy utility 
that properly dumps out any data returned 
by NtQuerySystemInformation() without 
knowing in advance what this data is like. 
Unfortunately, the function does not tell 
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you how large your buffer should be if it 
returns with STATUS_INFO_LENGTH_MIS- 
MATCH. (Closer examination of the imple- 
mentation in ntoskrnl.exe reveals that the 
kernel also doesn’t know beforehand how 
much data will be returned— it usually bails 
out as soon as it’s about to write beyond 
the end of the buffer and discards any wait- 
ing data.) 

Microsoft programs that have to deal with 
lists of the variable-size type (ike, for in- 
stance, process lists) use a trial-and-error 

strategy to find out the correct buffer size: 
- They start off with a reasonable initial size 
(32 or 64 KB, for instance), and increase 
the size by a fixed increment if Nf- 
QuerySystemInformation() fails with a 
length mismatch status. In the ntlib.c/ntin- 
fo.c demo programs (available online), I 
use multiples of 64 KB, which seem to be 
a good choice based on my empirical tests. 

The situation is further complicated by 
some info classes accepting larger buffers 
than required, while others insist on re- 
ceiving a properly sized buffer. Some might 
report success although the buffer is actu- 
ally too small, and still others fail to report 
the number of returned bytes (for exam- 
ple, set *pdSize = O). Therefore, ntqsi.c uses 
a somewhat clumsy, but successful, ap- 
proach to determine the correct size: 


e It starts with an initial size that is cho- 
sen large enough to hold even very 
large lists (256 KB) and calls NtQuery- 
Systemlnformation() repeatedly, decre- 
menting the size by 1 until the buffer 
size is 0 or a status code other than STA- 
TUS_INFO_LENGTH_MISMATCH is de- 
tected. 

e If the status code is STATUS. SUCCESS 
and the reported size is nonzero, this size 
is assumed to be the correct buffer size. 

e If the status code signaled success and 
the reported size is 0, the program starts 


with a 1-byte buffer and keeps on grow- 





ing it while calling MQuerySystemInfor- 
mation() until a status other than STA- 
TUS_INFO_LENGTH_MISMATCH is re- 
turned. If it’s STATUS_SUCCESS, the 
current buffer size is the desired value. 


All info classes returning variable-size 
data succeed after the first call to Nf- 
QuerySystemInformation() and set the re- 
sult size properly. Listing One is a sam- 
ple implementation. Many info classes 
returning fixed-size data blocks succeed 
as soon as the buffer size is decremented 
to the correct size, and set the result size 
to this value. The second pass is only 
needed for some odd info classes that be- 
have unlike the standard. 

Of course, this heuristic is time con- 
suming for info classes that return just a 
couple of bytes and require an exact size 
specification. However, the latency is in 
the range of just a few seconds on Pen- 
tium machines. (On a 400-MHz Pentium 
II, it takes 16 seconds to dump the entire 
currently valid info class range of 0 
through 45.) Anyway, this is the only gen- 
eral approach I know of that returns cor- 
rect results for virtually all info classes de- 
fined for NT 4.0 SP5. Of course, you can 
optimize the algorithm by using @ priori 
knowledge about the info classes, but I 
wanted to create a program that’s ex- 
pected to work reliably even on future 
versions of NT without change. 

The general command-line format of 
ntqsi.exe is: ntgsi { [+p |-p/ /<error limit>] 
[ <class> | [<class#1>/-[<class#n>] ] }. 


Pointer Relocation 

In the most simple case, you can enter, 
say, ntqsi 18, and get a hex dump con- 
taining some familiar looking Unicode text. 
Obviously, this is some information about 
your page-file configuration. Example 2 
is a sample printout. Now try the same 
command with +p inserted before the 


Example 1: The prototypes of NtQuerySystemInformation() and 


NtSetSystemInformation(). 





Example 2: lhe results of the command of ntqsi 18. 
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like the hex dump in Example 3. There’s 
not much difference, except for the 4 bytes 
at offset 0x14 (marked by “=” characters 
in between). As you'll see, this is the 
“Buffer” member of a UNICODE_STRING 
structure (that is, a pointer to a Unicode 
string) holding the path of a page file. 

If NtQuerySystemInformation() returns 
pointers inside the result data, they always 
point to addresses inside the caller’s buffer. 
This ensures that all pointers remain valid 
as long as the caller is working with the re- 
turned data, and that deallocating the data 
buffer also frees all data blocks referenced 
by those pointers. ntqsi.exe is clever enough 
to detect pointers inside the returned data 
stream and marks them with “=” characters 
between the 4 bytes forming the pointer 
(see Examples 2 and 3, offset 0x14). More- 
over, it subtracts the address of its internal 
buffer from the pointer, thus normalizing it 
to become a 0-based offset into the data. 
If you read out the value at offset 0x14 in 
Example 3, you get 0x00000018, and that’s 
exactly where you'll find a sequence of Uni- 
code characters making up the file path 
“D:\pagefile.sys.” 

Of course, ntqsi.exe lets you specify 
ranges of info classes, like nigsi O— 45, or 
sets like nigsi 1 2 5 3. In a range specifi- 
cation, you can omit the starting or end- 
ing class, or even both. Omitting the first 
part of the pair defaults to 0, omitting the 
second part means “go on forever.” (There 
is an upper limit, imposed by the number 
of values a DWORD variable can take.) 
Because the latter case might produce tons 
of garbage as soon as the last info class 
is transcended, it’s wise to use the Ker- 
ror limit> option here. For instance, us- 
ing nigsi /7 37- will dump all info class- 
es starting at 37, and stop after seven 
consecutive errors have occurred. 

ntqsi.exe attempts to find out which 
members of a structure are actually used 
by the system. Therefore, it fetches the 
data twice after presetting the buffer with 
certain bit patterns. On the first run, it uses 
0x55 for every byte, and OxAA on the sec- 
ond run. Every byte position that contains 
0x55 after the first and OxAA after the sec- 
ond call is considered to be unused. Those 
bytes are marked by a double dash in the 
hex output. 


The NT Kernel Debugger 

Although it’s interesting to study hex dumps 
of MQuerySystemInformation(), it would 
be more exciting to know what all those 
bytes are about. That’s really difficult, and 
it took me several weeks to compile 
enough information to get a basic under- 
standing of the returned data. If you ever 
try to gain information about undocu- 
mented NT interfaces and don’t happen to 
have access to the source code, there’s 
probably no way out other than using the 
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(continued from page 42) 

NT Kernel Debugger (KD), along with the 
symbol files on your NT setup CD (ook 
for \support\debug\i386\symbols). 

The KD comes in two flavors: the GUI 
version windbg.exe shipping with the Plat- 
form SDK, and the console-mode version 
i386kd.exe (or alphakd,exe, mipskd.exe, 
or ppckd.exe, if you happen to use a plat- 
form other than Intel x86) included on the 














Example 3: The results of the command of "ntqsi +p 18." 


NtQuerySysteminf ormation a 8) > 6h / 


08 09 OA OB OC OD OE OF | 91234567 


Offset! 00 01 02 03 O4 05 06 O7 : : 89ABCDEF _ 
j 5 : : Eeeeas ee Beaetones : { : 

20000 | OO OO 02 00 OO 8B 20 OO : 66 OB OO OO 66 OR OO 00! ....... : £...£... 

00010 | 26 00 28 OO 18=00-=00-00 : 5C 00 3F 00 3F OO 5C OO | &.( -\77). 

00020 | 44 OD 3A OD 5C OO 70 OO : 61 OO 67 OO 65 OO 66 OO | D. ae 

00030 | 69 OO 6C OO 65 08 2E OO : (a. a. 


73 00 79 O8 73 00 00 00 


NT setup CD (\support\debug\<plat- 
form>). I recommend using i386kd.exe, 
because it features a much more power- 
ful instruction set than windbg.exe. In Jn- 
side Windows NT, Second Edition (Micro- 
soft Press, 1998), David A. Solomon gives 
detailed instructions on how to set up 
i386kd.exe and the symbol files, using the 
information from a forced crash dump sys- 
tem image. The basic steps are: 
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1. Enable creation of a crash dump image 
in the Control Panel (System applet, 
“Startup/Shutdown” tab). Also, be sure 
to increase your page file size (twice 
the amount of physical memory is a 
good value) so you won’t run into a 
virtual memory problem later. 

. Force a crash dump by killing the “Client 
Server Runtime Process” csrss.exe, which 
is the module containing the Win32 sub- 
system. (You need kill.exe from the NT 
Resource Kit or an equivalent utility that 
is able to grab enough privileges to ter- 
minate a system service.) 

3. Reboot so the crash dump image is 
copied from the page file to the file 
specified in the Control Panel (%sys- 
temroot%\memory.dmp by default). 
Note that your system will run quite 
low on virtual memory after doing that, 
so you should reboot once more. 
(That’s why I recommended increasing 
the page file size in the first step.) 

4. Set the _NT_SYMBOL_PATH environ- 
ment variable to the base directory of 
the symbol files. This directory contains 
a couple of subdirectories, like “dll,” 
“drv,” “exe,” and the like. 

5. Issue the command i386kd -z <crash 
dump image>, where “<crash dump im- 
age>” is the full path of the crash dump 
file created before. 


NO 


Having done this, you can explore a 
snapshot of the system that was valid at 
the time you were killing csrss.exe. You 
can’t access some system data structures 
in user-mode memory, like the process 
and thread environment blocks. You need 
a live system to explore those. Now, at 
the kd> prompt, you can enter the de- 
bugger commands of your choice. 

There are both built-in commands and 
external bang commands prefixed by a “!” 
character, which are processed by an ex- 
tension DLL. i386kd.exe uses the exten- 
sion DLL kdextx86.dll by default (kdex- 
talp.dll, kdextmip.dll, and kdextppc on 
the other platforms). Those DLLs export 
a couple of APIs that are named like the 
corresponding bang commands they han- 
dle. Enter /belp at the kd> prompt to get 
a list of bang commands supported by the 
extension DLL currently in use. 


Retrieving Internal NT Symbols 

Probably the most useful kernel debug- 
ger command is /n (list nearest symbol). 
You can specify either a hex address or 
a symbol name after the /n token, and 
you'll get the name and address of the 
symbol whose address is below the spec- 
ified one and has the least distance from 
it. It also displays the address and name 
of the next symbol, so you get an idea 
of how large the data addressed by the 
symbol actually is. (Some symbols are 
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obviously missing, so the distance be- 
tween two symbols can be misleading.) 

When you're exploring NT internals, you 
usually don’t have more than a disassem- 
bly listing at hand, which will not contain 
more symbolic information than is export- 
ed from the NT system modules. The /n 
debugger command helps you convert 
opaque binary addresses into meaningful 
symbols that might suggest what this ad- 
dress is used for. Specifically, you can look 
up the names of all internal functions used 
by the code you’re currently examining. (If 
you're still looking for a good Win32 dis- 
assembler, you might want to contact Jean- 
Louis Seigne at jls@chez.com who has writ- 
ten a great one.) 

Another useful KD command, x nt/*, 
yields a complete list of symbols defined 
inside the NT kernel. And finally, the com- 
mands db (dump BYTEs), dw (dump 
WORDs), and dd (dump DWORDs) let 
you see hex dumps of memory regions in 
the appropriate number format. 

Although the APIs NtQuerySystemIn- 
formation() and NiSetSystemInformation() 
are exported by ntdll.dll, the actual im- 
plementation is located in a different mod- 
ule: ntoskrnl.exe. As Example 4 shows, 
the implementation of those APIs inside 
ntdll.dll is trivial. The secret of using INT 
2Eh is that it serves as a gate from the pro- 
cessor’s user mode to kernel mode, so the 
handlers can execute with enough privi- 
leges to access kernel-mode code and 
data. (On other CPU platforms, different 
but equivalent means are provided to 
achieve this effect.) 

On the other side of the INT 2Eh gate, 
the NT operating system kernel 
(ntoskrnl.exe) is waiting. It uses the in- 
dex value in register EAX to look up an 
entry in the kernel’s service descriptor 
table. This entry contains the informa- 
tion that’s needed to copy the required 
number of bytes from the caller’s stack 
(addressed by EDX) to a system stack, 
and to call the associated handler. Un- 
der Windows NT 4.0 SP3, the handlers of 
NtQuerySystemInformation() and N¢tSet- 
SystemInformation() happen to be locat- 
ed in the PAGE section of ntoskrnl.exe at 
0x801558c2 and 0x80156a48, respective- 
ly. You can use the kernel debugger’s /n 
command to verify that. 


System Information Classes 

In Windows NT Version 4.0 SP5, Nr- 
QuerySystemInformation() and NtSetSys- 
temInformation() recognize info class val- 
ues in the range of 0 up to 45. However, 
not all info classes will return a success 
status. The info classes can be put in one 
or more categories of the following: 


e Read-only info classes (not to be used 
with NiSetSystemInformation( )). 
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e Write-only info classes (not to be used 
with NtQuerySystemInformation(). 

e Info classes only available in the 
“Checked Build” of Windows NT. 

e Invalid info classes (either defunct or 
superseded by other info classes or sys- 
tem APIs). 
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To keep things simple, I deal almost ex- 
clusively with the first category. Table 1 lists 
all info classes I’ve been able to identify. 


A Custom Header File for ntdll.dil 


To save you from the hassle of directly 
interfacing to ntdll.dll, ’ve designed the 
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library ntlib.dll, which contains high-level 
APIs to load and parse the more complex 
data sets returned by NtQuerySystemInfor- 
mation(), as well as several utility func- 
tions for device-driver management, mem- 
ory size, and time conversion. It also 
features a pore command-line parser 
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with full-blown option management and 
command help support. The associated 
utility program ntinfo.exe demonstrates how 
to make use of it. The source code of both 
programs is available electronically. 

The main problem with ntdll.dll is that 
neither the Win32 Platform SDK nor the 
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Table 1: Available Sion er Classes. 
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NT DDK provide appropriate header files 
that can be used in user mode applica- 
tions. The main DDK header file ntddk.h 
is designed for kernel-mode drivers, which 
don’t rely on the windows.h file required 
for Win32 applications. Hence, Microsoft 
didn’t care to make them compatible. If 
you ever try to include both of them in a 
single project, the compiler will emit tons 
of errors and warnings. To remedy this, 
I’m using the same approach as Matt 
Pietrek in his MS7 articles: Create my own 
header file that contains just enough def- 
initions to be able to interface to ntdll.dll, 
while not sacrificing windows.h compat- 
ibility. Now, what name would be most 
appropriate for a file like that? Yes, of 
course: NtDIL.h. 

NtDll.h is designed to be modular. It 
defines macros, constants, and structures 
that are used in almost any part of the 
NT system. Near the end of the file, it 
yanks in the files NtObj.h, NtTeb.h, and 
NtSys.h. Only the latter is included in the 
online package and contains all defini- 
tions related to NtQuerySystemInforma- 
tion() and NtSetSystemInformation( ). 
The others are just stubs. The real things 
contain so much undocumented stuff 
about NT objects in general and 
thread/process objects in particular, that I 
decided to hold them back for a separate 
article on NT objects, hopefully to be 
available soon. 


Displaying Structured 

NT System Information 

The general command-line format of ntin- 
fo.exe is: ntinfo { <option> { [<parameter>] 
//. A command line may contain several 
options, which consist of a prefix charac- 
ter (“-” or “/”) and a mnemonic name, such 
as -pagefile or -process. The option name 
can be abbreviated, provided that the re- 
maining characters still identify it unam- 
biguously. Hence, ntinfo -p is invalid, be- 
cause it matches both -pagefile and 
-process. However, ntinfo -pa and ntinfo 
-pr are okay. Please note that ntinfo.exe 
cares to list all matching options in the 
case of an ambiguous specification, using 
the command-line parser included in 
ntlib.dll. 

Some options don’t expect additional 
parameters, while others can take one or 
more optional parameters; see Table 2 
for a summary of options. For instance, 
the -pagefile option takes no parameters 
and always lists all installed page files, 
while the output of the -process option 
can be filtered by specifying one or more 
process IDs. 

Some options display just a subset of 
the available information by default, and 
you have to specify “*” or “all” to get the 
complete data. The -object option belongs 
to this category. Still others accept a this 
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There’s no need to panic at release time. 


Would you believe that 


Insuret++* can automatically find your errors 
at the source code level? 
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program. You can then zero in on a problematic area and run 
Insure++ again in its default mode to receive more detailed error 
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None Basic system and processor configuration. 

None CPU identification and usage. 

None Time-of-day, time zone, and system time 
information. 

None Memory usage. 

None Cache usage and performance. 

None Page file configuration. 

| +lall] List of ERESOURCE locks. 

No parameters All locks where at least one counter is nonzero; 

*/all: all locks, regardless of counter values. 

[*lall] { [thisl<ID> ] } List of modules. 

No parameters or v/all All kernel-mode drivers; this: user-mode 
modules loaded by ntinfo.exe; <ID1> ... 
<IDn>: modules matching one of the 
specified IDs. 

-handle [*lall] { [thisi<PID>] } List of open handles. 

No parameters or */all: All open handles held by all processes: this: all 
handles held by ntinfo.exe; <PID1> 
<PIDn>: all handles held by the specified 
processes. 

[*lall] { [thisl<PID>] } List of LPC port objects. 

No parameters All named objects referenced by all processes; 
*/all: all objects — named and unnamed — 
referenced by all processes; <PID1> ... 
<PIDn>: all objects referenced by the 

specified processes. 
-process [**lall] { [thisl<PID>] } List of processes. 

No parameters or */all All processes in the system; this: process info 
about ntinfo.exe; <PID1> ... <PiDn>: info 
about the specified processes. 

-thread [*lall] { [thisl<PID>|<TID>] } List of threads. 

No parameters or */all All threads in the system; this: list of threads 
Started by ntinfo.exe; <PID1> ... <PiDn>: 
threads started by the specified processes; 
<IID1> ... <T1Dn>: info about the specified 
threads. 

-lookaside [*lall] { [<tag pattern>] } List of lookaside lists. 

No parameters or */all: All lookaside lists in the system; <tag pattern1> 
... stag patternN>: all lookaside lists with a 
tag that matches at least one of the specified 
patterns (case sensitive). 

None Ntinfo command help. 

None Ntinfo command help. 





Table 2: Options available from ntinfo.exe. 


(continued from page 40) “*” or “all” is specified with “this” or an 
parameter that defaults to the current pro- individual ID or pattern, it means that the 
cess, and which is, of course, ntinfo.exe result should contain all items except those 
itself. For example, ntinfo -handle this lists — specified in addition to “*” or “all.” 
all handles currently held by ntinfo.exe. It's remarkable that this undocument- 
You can also specify a negative filter: If ed system interface hasn’t changed very 
: z 3 much in Windows 2000. The latest Beta 
available at this writing (Release Candi- 
date 1) adds two more info classes, and 
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: ae introduces minor changes in a few data 
irectory ; ae . ; 
ra . Syrcle nk structures, For example, the ntinjo op 
4 Token tion '-thread" displays garbage, "-look- 
a0) 5 Process aside" fails on some entries, and "object" 
: es might throw an exception while trying to 
-”= 8 Event Pair display the new "WaitablePort" object list. 
—- cS) ei However, the remaining options appear 
emaphore ; : 
3 Timer to work correctly. 
Profile ; 

—_ Window Station Conclusion 
ye" Heo The system information interface dis- 
Key cussed here and exposed by the ntdll.dll 
on API pair NtQuerySystemInformation( ) 

apter es. 

Coase: and NiSetSystemInformation() offers 
Device functions — querying system configura- 
Driver tion items and process/thread or system 
Coe. a module lists— no system utility devel- 





oper can live without. 


Table 3: Object type IDs used by the 
SystemHandleInformation class. DDJ 
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eerie. e.g 


e e 
Listing One 
NTSTATUS WINAPI QuerySystemInformation (SYSTEMINFOCLASS sic, 
PPVOID ppData, 
PDWORD pdData) 


PVOID pData; 
DWORD dData, n; 
NTSTATUS ns = STATUS_INVALID_PARAMETER: 


dData = @; 
if (ppData != NULL) 
{ 
n = @; 
while ((pData = LocalAlloc (LMEM_FIXED, dData += 0x1@@@@) ) 
!= NULL) 
{ 


ns = NtQuerySystemInformation (sic, pData, dData, &n); 
if (ns != STATUS_SUCCESS) n = @; 
if (ns != STATUS_INFO_LENGTH_MISMATCH) break; 
LocalFree (pData); 
J 

dData = n; 

if (pData != NULL) 
{ 


if (ns != STATUS_SUCCESS) 
{ 
LocalFree (pData); 


pData = NULL; 
dData = @; 
} 

J 

else 

{ 

ns = STATUS_NO_MEMORY; 

} 


*ppData = pData; 
} 


if (pdData != NULL) *pdData = dData; 


return ns; 
} 
e e 
Listing Two 
// 5: SystemProcessInformation 
// see ExpGetProcessInformation() 
// see also ExpCopyProcessInfo(), ExpCopyThreadInfo() 
typedef struct _SYSTEM_THREAD 
{ 
QWORD qKernelTime; // 10@ nsec units 
QWORD qUserTime; // 1@@ nsec units 
QWORD qCreateTime; // relative to 01-01-1601 
DWORD di8; 
PVOID pStartAddress; 
CLIENT_ID Cid; // process/thread ids 
DWORD dPriority; 
DWORD dBasePriority; 
DWORD dContextSwitches; 
DWORD dThreadState; // 2=running, 5=waiting 
KWAIT_REASON WaitReason; 
DWORD dReserved@1 ; 
} 
SYSTEM_THREAD ; 


[J ----------------------------- ee 
typedef struct _SYSTEM_PROCESS_INFORMATION 
{ 


DWORD dNext; // relative offset 
DWORD dThreadCount ; 
DWORD dReserved@1; 
DWORD dReserved@?2; 
DWORD dReserved@3; 
DWORD dReserved@4; 
DWORD dReserved@5; 
DWORD dReservedg@6; 
QWORD qCreateTime; // relative to 91-01-1601 
QWORD qUserTime; // 10@ nsec units 
QWORD qKernelTime; // 1@® nsec units 
UNICODE_STRING usName; 
KPRIORITY BasePriority; 
DWORD dUniqueProcessId; 
DWORD dInheritedFromUniqueProcessId; 
DWORD dHandleCount ; 
DWORD dReserved9@7 ; 
DWORD dReserved@8 ; 
VM_COUNTERS VmCounters; 
DWORD dCommitCharge; // bytes 
SYSTEM_THREAD ast []; 
} 
SYSTEM_PROCESS_INFORMATION; 
® e 

Listing Three 

void WINAPI DisplayProcesses (void) 
{ 
NTL_TABLE nt; 
PSYSTEM_PROCESS_INFORMATION pspi; 
TIME_FIELDS ce: 
DWORD a3 
if (NtlTableProcess (&nt) == STATUS_SUCCESS) 

{ 


printf (T("PID Org BP Th Hdls CommChrg WSetSize ") 
T("PFCount Start date and time Name\r\n")) ; 
pspi = NtlTableFirst (&nt); 
for (i = @; i < nt.dCount; it+) 
{ 
NtlTimeUnpack (&pspi->qCreateTime, &tf); 
printf (T("\r\n%3lu %3lu %21u %2lu %41lu %8lu %8lu ") 
T("%7lu %O2u-%02u-%O4u %O2u:%O2u:%O2u ") 
TINS s\"") , 
pspi->dUniqueProcessId, 
pspi->dInheritedFromUniqueProcessId, 
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pspi->BasePriority, 
pspi->dThreadCount, 
pspi->dHandleCount, 
pspi->dCommitCharge, 
pspi->VmCounters.WorkingSetSize, 
pspi->VmCounters.PageFaultCount, 
tf.Month, tf.Day, tf.Year, 
tf.Hour, tf.Minute, tf.Second, 
(pspi->usName.Buffer != NULL 
? pspi->usName. Buffer 
: L"Idle")); 
pspi = NtlTableNext (&nt, pspi); 
} 
NtlTableUnload (&nt) ; 


} 
return; 


} 


e e 
Listing Four 
// 11: SystemModuleInformation 


// see ExpQueryModuleInformation 
typedef struct _SYSTEM_MODULE 
{ 


DWORD dReserved@1; 
DWORD d@4; 
PVOID pAddress; 
DWORD dSize; // bytes 
DWORD dFlags; 
WORD wid; // zero based 
WORD wRank; // ® if not assigned 
WORD wi8; 
WORD wNameOffset; 
BYTE abName [MAXIMUM_FILENAME_LENGTH] ; 
} 
7 SYSTEM_MODULE ; 
typedef struct _SYSTEM_MODULE_INFORMATION 
{ 


DWORD dCount; 
SYSTEM_MODULE asm []; 

} 
SYSTEM_MODULE_INFORMATION; 


e e e 

Listing Five 

// 16: SystemHandleInformation 

// see ExpGetHandleInformation () 

typedef struct _SYSTEM_HANDLE 
{ 
DWORD dIdProcess; 
BYTE bObjectType; // OB_TYPE_* 
BYTE bFlags; // bits @..2 HANDLE_FLAG_* 
WORD wValue; // multiple of 4 


POBJECT pObject; 
ACCESS_MASK GrantedAccess; 


} 
SYSTEM_HANDLE; 
e e e 

Listing Six 

// 18: SystemPageFileInformation 

// see MmGetPageFileInformation() 

typedef struct _SYSTEM_PAGE_FILE_INFORMATION 
{ 
DWORD dNext ; // relative offset 
DWORD dTotal; // pages 
DWORD dInUse; // pages 
DWORD dPeak; // pages 
UNICODE_STRING usName; 
} 
SYSTEM_PAGE_FILE_INFORMATION; 

e e 

Listing Seven 

// 26: SystemLoadDriver (set mode only) 

// see MmLoadSystemImage() 

// user mode: STATUS_PRIVILEGE_NOT_HELD returned 

typedef struct _SYSTEM_LOAD_DRIVER 
{ 
UNICODE_STRING usImageFile; // input 
PVOID pBaseAddress; // output 
HANDLE hSystemImage; // output 
PVOID pEntryPoint; // output 
PVOID pDirectoryEntry; // output 


SYSTEM_LOAD_DRIVER; 
// 27: SystemUnloadDriver (set mode only) 
// see MmUnloadSystemImage() 
// user mode: STATUS_PRIVILEGE_NOT_HELD returned 
typedef struct _SYSTEM_UNLOAD_DRIVER 
{ 
HANDLE hSystemImage; // received via SystemLoadDriver 
} 
SYSTEM_UNLOAD_DRIVER; 
// 38: SystemAddDriver (set mode only) 
// see MmLoadSystemImage(), MmUnloadSystemImage() 
// user mode: SeLoadDriverPrivilege required 
typedef struct _SYSTEM_ADD_DRIVER 
{ 


UNICODE_STRING usImageFile; 


} 
SYSTEM_ADD_DRIVER ; 
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functionality with 
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Greg Winton 


ackaging code into shared libraries 
provides many benefits. For one thing, 
libraries provide a mechanism for 
reusing your code among two or 
more applications, thereby reducing devel- 
opment time. For another, multiple appli- 
cations can use the same library, reducing 
both the installed and in-memory footprint. 
The lifetime of a shared library is not strict- 
ly tied to that of its clients, so its state can 
be maintained across applications. Finally, 
architecturally, shared libraries reduce cou- 
pling between modules, which results in 
more stable and maintainable code. 
Although there is limited support from 
the CodeWarrior tools, a handful of arti- 
cles and white papers, and one example 
on the Palm web site, information on 
shared libraries for the Palm Computing 
Platform is still hard to come by. Conse- 
quently, in this article, I'll examine the Palm 
shared library model and present a shared 
library (available electronically; see “Re- 
source Center,” page 5) that you can use. 


The Palm Shared Library Model 

A shared library is a special type of database 
that provides services to clients; these clients 
may be either Palm applications or other 
shared libraries. These clients refer to the 


Greg is a director of software development 
for Bachmann Software and a contribu- 
tor to Palm Programming, by Glenn Bach- 
mann (Sams Publishing, Macmillan Com- 
puter Publishing, 1999). Greg can be 
contacted at greg@bachmannsofitware.com 
or at http://www.bachmannsoftware.cony. 
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PalmOS 


shared library by its reference number, and 
invoke functions published by the library. 

From the client side, use of a shared li- 
brary is straightforward. The client loads the 
shared library into memory using SysLib- 
Load. If the library has already been load- 
ed, this function will fail— the client should 
then call SysLibFind to locate the loaded li- 
brary. System libraries, such as Serial/Mgr 
and IrMgr, are always loaded, so clients of 
these libraries can skip the load step. Cus- 
tom shared libraries are not initially load- 
ed, and thus must be loaded by the client. 





ae 


On success, SysLibLoad and SysLibFind 
both return a library reference number, 
which must be used for all subsequent calls 
to the shared library’s functions. The con- 
vention is that the first function to be called 
must be the Open function. Then the client 
may call the rest of the library’s API. 

When the client is finished with the 
shared library, it calls the library’s Close func- 
tion. It may also need to unload the library 
using SysLibkemove. It is recommended that 
Close should return success if it can be un- 
loaded, or an error if it still has active clients. 

In conventional Palm development sce- 
narios, functions are linked in as you would 
expect with any other platform. It is when 
you invoke functions defined in shared or 
system libraries that things get interesting. 
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If you look at any system header, you 
will notice that every function declaration 
is followed by the SYS_TRAP macro. The 
argument of this macro is the function trap 
identifier. This declaration syntax causes 
the function’s arguments to be placed on 
the stack and then invokes the system trap 
dispatcher. This dispatcher maps the trap 
identifier to a specific function call. 

The trap identifiers for system libraries 
are unique throughout the operating sys- 
tem. Shared library function trap identifiers 
are unique only within the library. As a 
shared library developer, you have to de- 
fine the trap identifiers for your library, and 
implement the mechanism whereby traps 
are mapped to function calls. 

The system provides three services for 
shared libraries: 


e The system trap dispatcher dispatches 
calls by library reference number to the 
appropriate library. When a client in- 
vokes a function by a trap identifier, it 
generates an exception. The trap dis- 
patcher handles this exception by 
putting the address of the appropriate 
function onto the stack. When the ex- 
ception returns, program execution is 
transferred to this address. 

e For each shared library open in the sys- 
tem, there is a corresponding entry in 
the system library table. This entry holds 
both the library’s dispatch table and the 
memory allocated by the library for its 
global memory. 
The library reference number is assigned 
by the system when the library is load- 
ed for the first time. The client retrieves 
this number either from SysLibLoad, 
which loads the library, or SysLibFind if 
the library is already loaded. This refer- 
ence number must be the first parame- 
ter to every shared library call; the sys- 
tem trap dispatcher uses it to locate the 
correct library. 


The shared library provides the mecha- 
nism by which trap identifiers are translat- 
ed into function calls. There are several 
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#include <iostream.h> 


int sum( int a[], intn ) 


{ 
anc: i.; 
int s = 0; 


for( i=0, i<n; 
s += a[il]; 
return s; 


} 
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at4; ) 


int a[10] = { 0, ‘i 2; 2; 4, 5, 6, 7, 8, 9 ee 


int main () 


{ 


cout << "the sum is " << sum(a,10) << "\n"; 


return 0; 


Se 


ean 
ce 


The programmer expected this program to print that the sum is 45. Instead it prints that the 
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parts to this. There is the definition of the 
trap identifiers and the associated func- 
tion declarations. The installation entry 
point is invoked when the library is load- 
ed. It informs the system of the library’s 
dispatch table and initializes memory re- 
quired by the library. 

The installation entry point is called 
«_Startup__,” and must be the first func- 
tion in the library’s link order. It returns 0 
for success, or a negative error code. Sys- 
LibLoad returns the error code to the client 
application. The system passes the library’s 
entry in the system library table as a pa- 
rameter to this function; the library sets the 
dispatch table attribute of this structure. 

The library dispatch table contains a list 
of all routines in the library. This is a 
lookup table used by the system trap dis- 
patcher. 


Implementing a Shared Library 

There are three components in a shared 
library: the API declarations, API imple- 
mentation, and dispatch table. 

The shared library publishes a header 
file describing the API to clients. Function 
declarations specify how the client invokes 
the API, including parameter information 
and return types. Trap identifiers allow the 
system to invoke the function using the 
trap mechanism on which the PalmOS re- 
lies. Result codes provide an expected set 
of errors for which the client should check. 
Domain-specific structures and constants 
provide additional information required 
by the custom portion of the API. 

Every shared library must publish four 
standard functions: 


e The client must call the Open function 
first. It allows the shared library to ini- 
tialize any resources it needs. No other 
API functions can be called prior to this 
function. 

e The client calls the Close function last. It 
allows the shared library to release any 
resources it is holding. Once this function 
is called, the library is in an invalid state; 
it must be reopened before it can be used. 

e The operating system calls the Sleep 
function before the device enters sleep 
mode. It allows system-level libraries to 
shut down hardware components to 
conserve power. 

e The operating system calls the Wake 
function when the device returns from 
sleep mode. It allows system-level li- 
braries to reenable any hardware com- 
ponents that were shut down when the 
device entered sleep mode. 


The shared library also publishes domain- 
specific functions. As a rule, these functions 
must be invoked after Open and before 
Close, to make sure the library has valid 


resources. The one exception is a func- 
tion that retrieves the API version of the 
library, which may be invoked before 
Open to ensure compatibility. 

If you examine the PalmOS SDK head- 
ers, you'll notice that every function dec- 
laration includes the SYS_TRAP macro: 


Err SerOpen(UInt refNum, UInt port, ULong 
baud) SYS_TRAP(sysLibTrapOpen); 


For the CodeWarrior compiler, this dec- 
laration expands to a Metrowerks exten- 
sion called “opcode inline” syntax: 


Err SerOpen(UInt refNum,UInt port,ULong 
baud)={m68kTrapInstr+sysDispatch 
TrapNum,trapNum} 


The opcode inline syntax lets you spec- 
ify the 680x0 opcodes for the function's 
implementation. When you call an opcode 
inline function, the compiler replaces the 
function calls with the specified opcodes. 
This feature supports calls through the 
680x0’s “A-Trap” mechanism; it generates 
the exception, which is handled by the 
PalmOS’s system trap dispatcher. 

Invoking a call to a shared library func- 
tion is a two-step process. For client code, 
the function declaration expands into a call 
to the A-Trap mechanism. This mechanism 
uses the library’s reference number to iden- 
tify the appropriate entry in the system li- 
brary table; the library’s reference number 
must be the first parameter to any shared 
library function. Using the trap identifier as 
an offset into the entry’s dispatch table, the 
appropriate function is called. 

If this all seems like a lot of work, re- 
member that the compiler takes care of it; 
your job is to declare the trap ID enu- 
meration, and to make sure you use the 
right trap for each function declaration. 

The system defines traps for the required 
Open, Close, Sleep, and Wake functions. You 
define the rest, starting with sysLibTrap- 
Custom, and incrementing sequentially. 

The public API also contains the errors 
that the functions may return. These are 
based on the appErrorClass, which is de- 
fined in SystemMer.h. The library functions 
should not return results that are not either 
a success or one of the API-defined errors. 

In addition to the functional API, there 
may be structures or constants defined by 
the library. For example, a printing library 
might publish a font structure and con- 
stants for bold, italic, or underline. 

The functions described in the public 
header are implemented in a standard C 
file. In this module, the SYS_TRAP macro 
is disabled so that the function declarations 
evaluate to standard C declarations. 

The Open function is responsible for al- 
locating memory for this information and 
storing it in the system library table entry 
for the shared library. Once this is done, it 
performs any domain-specific initialization. 
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The Close function is responsible for re- 
leasing memory allocated for the library’s 
globals, and removing this value from the 
system library table entry. Any domain- 
specific clean up should be done prior to 
releasing this memory. By convention, this 
function returns 0 if the library should be 
removed, or an error code indicating that 
the library is still in use by other clients. 

The Sleep function handles notification 
from the system that the system is about 
to shut down. This notification lets system- 
level libraries shut down hardware com- 
ponents to conserve power. 

The Wake function handles notification 
from the system that the system is about 
to wake up again. This notification lets 





Table 1: Typical dispatch table. 
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system-level libraries reenable hardware 
components that were shut down when 
the system went to sleep. 

Because these functions are invoked 
by system interrupts, they may only use 
interrupt-safe system services and must 
not take a long time. 

Domain-specific API functions depend 
on the global data allocated in the Open 
function, and so should only be invoked 
between Open and Close. These functions 
follow a standard pattern. First, they re- 
trieve the library’s global data. Then, they 
perform the domain-specific task they pro- 
vide. Finally, they release the global data. 

There are exceptions to this rule. For ex- 
ample, many libraries provide a function to 
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retrieve the API version of the library. Be- 
cause this function does not rely on any 
global information, it may be called be- 
fore the Open or after the Close function. 
Indeed, it probably should be called pri- 
or to Open to ensure compatibility be- 
tween the client and the library. 

The dispatch table implementation is pro- 
vided by the /nstall and DispatchTable func- 
tions. The system invokes Install as part of 
SysLibLoad; this function invokes Dis- 
patchTable, which returns the address of 
the dispatch table. This address is stored in 
the library’s entry in the system library table. 
DispatchTable is coded in assembler and 
includes the declaration of the dispatch table 
as well as the code that returns its address. 

The first section of the dispatch table is 
an array of offsets from the beginning of 
the dispatch table. The first offset points 
to the library name. This string is stored 
at the very end of the table. The remain- 
ing offsets point to entries in the next sec- 
tion, which is an array of jump instruc- 
tions: The second offset points to a jump 
to the Open function; the third, a jump to 
the Close function; and so on. The library 
name is stored as a null-terminated string 
at the end of the jump table. 

For example, say that you had a shared 
library named “FooLibrary,” with only one 
function —Foo. The dispatch table would 


look like Table 1. The system executes li- 
brary traps using this table. Using the trap 
identifier as an index into the first section 
of the dispatch table, it retrieves the off- 
set to the jump instruction corresponding 
to the trap. The system moves to this off- 
set in the dispatch table and executes the 
next instruction. This instruction is a jump 
to the appropriate shared library function. 

As Figure 1 illustrates, this is conceptu- 
ally no different from the implementation 
of virtual tables under C++. The client in- 
vokes a function that is mapped to an off- 
set in a function table. Using this offset, the 
compiler retrieves the address of the func- 
tion to execute. The main difference is that 
in C++, the compiler takes care of all this 
plumbing transparently; with shared li- 
braries, we must do it ourselves. 

This may seem more complex than it 
actually is. The dispatch table in the sam- 
ple library I provide here (available elec- 
tronically) can be copied and does work. 
There are a few things to remember: 


e The order of the trap identifiers as de- 
clared in the public header must match 
the order of the respective functions in 
the jump table. 

e There is a macro defined in the sample, 
numTraps. This must be equal to the 
number of public functions declared. 





e There must be one entry in the dispatch 
table for each public function. 

e The standard functions must appear first 
in the jump table and in the correct or- 
der: Open, Close, Sleep, and Wake. 


If your library is crashing, or the wrong 
functions are being invoked, these are the 
first things to check. 

And just in case shared libraries weren’t 
daunting enough, there are some rules 
and caveats you should be aware of: 


e Do not use static and global values. Shared 
libraries are stored on the device as re- 
source databases. The memory occupied 
by the library is, therefore, on the storage 
heap. By default this memory is protect- 
ed; it can only be written to with the ap- 
propriate DataMgr function. A down- 
stream effect of this is that all global or 
static values are, effectively, read only. 

You could use static or global variables 
in your shared library. But if you do, you 
need to allocate a writeable data segment 
for them, and initialize this data segment 
by hand. When you enter a library func- 
tion, you need to set the A4 register to 
point to this data segment. When you exit 
the function, you need to set it back to 
its original value. You also need to do 
this “A4 magic” before calling certain SDK 
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functions, such as MemHandleNew. In 

general, it’s best not to use them at all. 
e Install linkage. A shared library’s Install 
function is actually a macro for the li- 
brary entry point, __Startup__. The sys- 
tem expects this function to be the first 
entry point into the library. Therefore, 
this function must be first in the linkage 
order. We put it in its own module, and 
list that module first in the Segments 
panel of the project. 
Library resources. Palm application re- 
sources are opened with the application 
at all times. If you want to show a form, 
or raise an alert, it is a straightforward 
task. Resources associated with a shared 
library are not left opened with the li- 
brary; the library must specifically load 
its resource database. Consequently, it 
must unload the database as well. 
Debugging. Bugs are a fact of life in soft- 
ware development. As a result, so are de- 
buggers. Unfortunately, the CodeWarrior 
debugger does not currently support trac- 
ing into shared library code. This support 
is rumored to be coming in an upcom- 
ing release. Until this support is available, 
you must embed code in your library to 
display errors, variable values, and so on. 
You can use alerts, error messages, or the 
emulator logging facility. On some pro- 
jects, we have created a debug message 
console on the Palm device. 


A Shared Library Example 

The sample shared library I present here 
lets the client application store and retrieve 
a text string. It also provides a function to 
retrieve the library’s API version; this is 
useful to ensure version compatibility be- 
tween application and library. The library 
also implements the four standard shared 
library functions— Open, Close, Sleep, and 
Wake. While the actual functionality is 
straightforward, this sample provides a 
framework onto which you can build your 
own shared libraries. 


Summary 

Shared libraries are not hard to create. 
Conceptually, they are function tables that 
follow specific rules. The bad news is that 
these rules are not intuitive or apparent 
to the first-time programmer. The good 
news is that once you've created your first 
library, you can use it for a template and 
stop worrying about these rules. The best 
news is that you can just use the sample 
provided with this article. 

Used correctly, shared libraries will re- 
pay the investment with greater code reuse, 
increased application stability, and reduced 
resource requirements. You might think of 
them as peace of mind, encapsulated. 
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The Power lV 
Operating System 





A powerful OS 
for set-top boxes 





Morgan Woodson 


he PowerTV Operating System (PTVOS) 
from PowerTV (http://www.powertv 
.com/) is designed for set-top boxes 
and hybrid fiber/cable (HFC) networks. 
It consists of PowerCore, a real-time kernel 
and memory manager, and PowerModules, 
which are libraries for communicating on 
the network, tuning analog and digital chan- 
nels, and displaying TV-centric graphics. A 
resident application lives in flash memory 
for user settings, changing channels, and 
displaying a program guide. Other applica- 
tions can be downloaded from servers at 
the cable operator’s head end (servers, ATM 
switches, and radio-frequency equipment 
for a cable system). Applications can be writ- 
ten in C and C++ to the PTVOS APIs, or in 
PersonalJava with Java Native Interface JND 
methods for TV operations. An HTML en- 
gine can be called to render HTML pages. 
The first commercial deployment of 
PTVOS is on the Explorer 2000— the 
first digital cable box (see Figure 1) from 
Scientific-Atlanta (http://www.sciatl.com/), 
PowerTV’s parent company. Scientific- 
Atlanta is presently building 500,000 box- 
es per quarter and has installed over 100 
digital head ends with over 40 launched 
to paying customers. 


Typical Environment 
The processor on the Explorer 2000 is a 
54-MHz microSPARC core. There is 4 MB 
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of flash for the OS and resident applica- 
tion and from 2 MB to 8 MB of DRAM for 
downloaded applications and data. It is a 
basic core with no MMU or FPU and only 
a small L1 cache. 

The media processor has an MPEG-2 
video decoder and a graphics compositor 
capable of 16-bit color. There are two 
tuners. The in-band tuner works from 57 
MHz (EIA channel 2) and up, tuning 6- 
MHz wide bands carrying either legacy 
analog channels or 27 Mbits/sec. QAM64 





modulated data containing MPEG-2 pro- 
grams or data. The out-of-band tuner 
works below 57 MHz, tuning channels that 
are QPSK modulated, carrying 1.5 
Mbits/sec. each. Out-of-band is two way 
and has the advantage that data can be 
received even when a video program is 
being watched. 

The system includes a smartcard slot, 
USB port, IR receiver for hand-held remote 
and wireless keyboard, and 10BaseT Eth- 
ernet port— but no mass storage. The de- 
velopment card has extra serial ports for a 
monitor and debugger, extra flash for a 
larger debug version of the OS, and a SCSI 
disk interface. PTVOS will soon be avail- 
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able on Scientific-Atlanta’s Explorer 3000 
(twice as fast as the 2000) and on a box 
for NTL in the U.K. Most boxes planned 
for the near future have performance from 
one to four times that of the Explorer 2000. 


PTVOS APIs 

PTVOS APIs all have a prefix (based on 
the name of the PowerModule) of two to 
four characters separated from the main 
function name by an underscore. For ex- 
ample, pd_NewGrafPort creates a new 
graphics viewport for the PowerDraw 
(hence pd) module. Structure names have 
the first letter capitalized; for instance, 
Pd_GrafPort, and constant names are pre- 
ceded by a k, as in RPd_VideoHeap. 

The PTVOS is intended for inexpensive 
32-bit RISC processors that do not have 
MMUs so threads do not have private 
memory space. There are 32 priority lev- 
els; threads of the same priority run round- 
robin. Thread switching is preemptive, but 
you are encouraged to write event-driven 
code that calls pk_NextEvent early and of- 
ten. Applications start with one thread 
each, but can spawn more as needed. For 
many applications, the real-time nature of 
the kernel will not matter. The indirect 
benefit is that deadline-critical tasks such 
as setting up new DMA transfers as old 
ones complete and decoding digital video 
can steal whatever CPU time they need 
without application intervention. 

Communication between threads and be- 
tween applications and the OS is done with 
Pk_Events in Pk_Queues (pk for Power- 
Kernel). Events can be delivered to a sin- 
gle queue or posted to all queues that have 
registered an interest in events of that type. 
Events are retrieved from queues with a 
family of calls. pk_NextEvent() returns con- 
trol to the kernel. When an event is present 
in the calling thread’s queue or the given 
timeout has been reached, the thread will 
be marked ready to run, and will run again 
according to its priority. 
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(continued from page 50) 

A complete C library is included with 
the OS, including malloc and free, but the 
memory manager provides more features 
if called directly. There are two basic 
heaps— system and video — but applica- 
tions can create subheaps. They can also 
allocate memory with handles, which helps 
prevent fragmentation. Handles are an in- 
direct reference to memory, which must be 
locked to get a real C pointer to the mem- 
ory. The handle refers to a small block of 
memory that contains the real pointer and 
is fixed in place. While unlocked, the real 
data can be copied around to facilitate com- 
paction. Compaction is automatically em- 
ployed when an allocation is about to fail 
but can also be called explicitly. 

Applications are built using GNU’s 
make, C/C++ compiler, and linker, sup- 
plied by Cygnus Support (http://www 
.cygnus.com/success/powertv.html). The 
OS is called through a dispatch table so 
applications need not be recompiled for 
different versions of the OS. Code in the 
OS may move around, but the entry points 
in the dispatch table remain constant. 


Window Manager and PowerDraw 
The Window Manager provides applications 
with controlled access to the graphics frame 
buffer that sits on top of the video plane. 
It also filters user input events. It doesn’t 
enforce a look-and-feel or provide window 
selection and rearrangement services, but 
does clip drawing giving the common 
stacked look to the windows. 

Window manager also provides event 
routing using the window stack. Events are 
sent to the top window, which can act on 
an event as necessary. It then either pass- 
es it on to underlying windows or consumes 
it so the other windows never see it. 

Windows that are partially or fully cov- 
ered by other windows are considered ob- 
structed. Drawing to fully obstructed win- 
dows is a no-op. Drawing to partially 
obstructed windows is handled by auto- 
matically repeating the drawing operation 
in each rectangle in a list that represents 


Figure 1: The Explorer 2000 unit 
connected to a TV. 
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the visible portion of the window. This is 
a bit slower than drawing to nonobstruct- 
ed windows, so you can decide only to up- 
date windows when they are on top. When 
a window does move to the top, it receives 
an expose everit and must redraw all of its 
content. The exception is private windows, 
which have an offscreen buffer exactly the 
size of the window. Drawing goes to the 
buffer, which is automatically copied on- 
screen when the window is exposed. They 
are more convenient at the expense of pre- 
cious memory. 

Another special type of window is the 
backdrop. This is a window the size of 
the entire screen, underneath all other vis- 
ible windows, that determines the color 
format and resolution of the screen. It also 
determines the overlay status of the screen. 
Overlay is a combination of chroma key- 
ing (makes one or a range of colors com- 
pletely transparent) and alpha blending 
(makes graphics translucent). It must be 
turned on once you have tuned a chan- 
nel to see the video. 

PowerDraw provides all the basic draw- 
ing primitives you expect: lines, rectan- 
gles, circles, ellipses, arcs, and polygons. 
PowerDraw is also used for rendering text 
through the Font Manager. New font en- 
gines can be plugged in, but the includ- 
ed bitmapped font engine and Agfa scal- 
able font engine are usually sufficient. 

While PowerDraw uses the concept of 
a drawing coritext, containing things like 
the current brush, linewidth, and antialiased 
drawing status (on or off), you aren’t forced 
to use it. The context is initialized to a de- 
fault state of no brush with a line width of 
one and once you have a Pd_GrafPort to 
draw into, you are all set. Lifting the float- 
ing cursor so it will not mangle graphics 
and iterating through clipping rectangle lists 
is handled automatically. 

Usually the grafport is obtained with a 
win_GetGrafPort call, but you might create 
your own grafports for a couple of reasons. 
First is creating off-screen grafports as 
scratchpads to render bits of graphics be- 
fore blitting them on screen using pd_Copy- 
GrafPort. Second is when you include your 
own previously rendered graphics. You 
might have an icon of a frog on a black 
background that you want to hop around 
the screen. A 24-bit Windows bitmap of the 
image is converted to a 16-bit image, then 
to an initialized const array in a C file us- 
ing PowerTV tools. You include this in your 
build and your code can allocate a grafport 
with no memory attached, then attach the 
array. Finally, blit the image on screen with 
the chroma key color set to black so only 
the foreground appears. 


Broadcast Filesystem 


Stream Manager provides a uniform way to 
read and write data from stream devices 
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such as FTP, HTTP, files, and serial ports. 
Its use is so pervasive through the system 
that even nonvolatile memory (NVM) and 
RAM or Flash can be accessed as streams. 
Streams to be opened are specified using 
the Uniform Resource Locator (URL) for- 
mat. Once open, sirm_Read, strm_Write, 
strm_Seek, and strm_Close are similar to 
their C standard library namesakes. Just like 
_read or fread, strm_Read takes the ID re- 
turned by strm_Open, a pointer to a buffer 
to stuff data into, and a count of bytes de- 
sired. An important difference is that you 
can choose to call open, read, and write 
synchronously, which does not return un- 
til done, or asynchronously, which returns 
- immediately and delivers an event to the 
given queue when the operation completes. 

strm_Get and strm_Set allow stream de- 
vices to have custom parameters. The first 
and second parameters specify the device 
and parameter and the third is a void*, al- 
lowing any kind of data to be passed or 
retrieved. 

While PTVOS includes a full TCP/IP stack 
and an RF interface for communicating over 
the cable to the head end, two-way net- 
working is discouraged. Two-way uses 1.5 
Mbits/sec. out-of-band channels. This band- 
width is shared by all subscribers in a neigh- 
borhood. While QPSK modulators can be 
added to a system to divide the subscriber 
base into smaller groups and provide more 
effective bandwidth, this is expensive. One- 
way is preferred except for applications that 
absolutely require it, like signing on to the 
system at boot up, buying pay-per-view 
movies, and sending e-mail. 

The downstream bandwidth is so high 
(27 Mbits/sec. per 6MHz-wide channel or 
1 to 2 Gbits/sec. total) that many appli- 
cations that require two-way on the In- 
ternet can use one-way networking in a 
cable system. A broadcast carousel is a 
stream of data on a channel that repeats 
as soon as it finishes. The data is divided 
into blocks, each with a unique address. 
Rather than requesting data using FTP or 
HTTP over TCP/IP, you request the blocks 
you want and the OS grabs them out of 
the stream when they come around. There 
is hardware filtering and DMA to help the 
OS so it does not have to consider every 
bit in a 27 Mbits/sec. stream. 

This system of simulating choice by 
snatching pieces of a large database ful- 
fills some of the ideas of push technolo- 
gy that was so hot a while back. Push may 
not be appropriate for the Internet, but 
works well on digital cable and on pagers. 
For instance, all of CNN’s current global 
and national stories and your local news- 
paper’s stories with pictures and weather 
maps can be rotated through in a few sec- 
onds. If the “Digital News” channel gets 
so large that response time suffers, it can 
be split into multiple channels. 

















































Looking for a way to 
integrate data from 
your mainframe, 
midrange or other 
host with the Web? 
ViewMax is a powertul 
tool that allows you 





fo preserve and extend 
your company’s current 
IT investments. 


Need quick access to your hosts? 
ViewMax INSTANTON capability converts your "green screens" 
to interactive HTML upon installation. 


Looking to build a simplified web GUI, integrate 
electronic commerce with your legacy systems, 
re-engineer your business process, or integrate 
multiple hosts to one interface? 

ViewMax is a rapid implementation solution that allows you to 
bring as much or as little legacy data as you require to the web 
— without any modification fo your host systems. 


ViewMax is designed for one purpose - TO WEB ENABLE 
YOUR ENTERPRISE FAST!! But don’t take 
our word for it. Put us to the test — we'll show 
you how your existing mainframe and 
midrange applications can be web 
enabled in a matter of weeks, not months — 
or years. For more information call 


888-846-3045 today. 
ViewMax is the complete solution to your host of problems. 


ViewMax. 


Web-to-Host Integration Tools 


www.modcomp.com/ viewmax 


MODGOMP ©1999. All rights reserved. MODCOMP is a registered trademark and ViewMax is a trademark of MODCOMP, Inc. 








The problem with a broadcast carousel 
is that it is a fairly low-level system. To 
tune a carousel, you must have control 
of the tuner, know what frequency the 
carousel is transmitting on, and know the 
number of the block of data you want. 
The broadcast filesystem (BFS) is built 
on top of broadcast carousel to make it 
look to an application more like a regu- 
lar disk filesystem. 

A listing of all files available through 
BFS is broadcast periodically with loca- 
tion and version information. The OS does 
not read the directory for the first time un- 
til a little while after it is booted. Appli- 
cations can be launched before this, so 
applications should check whether the di- 
rectory is available before opening BFS 
files; use strm_Get("bfs:///"", RBfs_Directory- 
Available, (void *) available);, where 
available is a pointer to a Boolean. 

Applications cannot read the BFS di- 
rectory directly, but the directory typi- 
cally contains many files that are of no 
interest to applications such as OS up- 
date images and security and configura- 
tion information. Filenames must be hard 
coded into applications, but a file con- 
taining the names of files a particular ap- 
plication needs could be part of a 
carousel. For instance, a news directory 
file would contain the filenames of all 
news stories, their associated picture file- 
names, and English names of the stories 
for presentation to users. 

Files on BFS have version numbers so 
an application can tell when they change 
using strm_Get with the value parameter 
set to RBfs_FileInfo for unopened files or 
RBfs_StreamInfo for open files. This al- 
lows the servers at the head end to pas- 
sively update files without having to 
broadcast messages to or process status 
requests from set tops. 





A Weather/Traffic 
Camera Viewer Application 
Interruption of national morning news 
broadcasts for local news/weather typical- 
ly happens every half-hour— and always 
when I am in the shower or in the kitchen 
getting breakfast. I commute on a road that 
snarls often and the weather in Cupertino, 
California, is often warmer than at my house 
in Santa Cruz, so I want this information. 
Booting a computer and dialing an ISP takes 
too long. What I want is to flip on the TV 
and get pictures from the local weather and 
traffic cams with no waiting. 

The sample application (see Figure 2) 
I present here (source code is available 
electronically; see “Resource Center,” page 
5) assumes a server at the head end that 
can retrieve pictures from web servers, 
scale them for display on TV, convert them 
to a format PTVOS can deal with, and 
move them to the BFS server. The app 
uses PowerTV’s proprietary IMG format— 
a 16-bit per pixel, uncompressed bitmap 
with a minimal header. A real-world app 
would use MPEG-2 I-frames because the 
hardware can decompress them in less 
than 1/60th of a second. This app scales 
well: A picture retrieved once from a web 
site can be broadcast to thousands of set 
tops without taxing bandwidth. 

The main features of the application 
are as follows: 


e A main loop that handles application, 
window, system, and user events. 

e A periodic update event. 

e A state machine per cam window. 

e One read thread launched for each cam 
window. 


An application has many responsibili- 
ties to be a cooperative member of the 
set-top society. It must acknowledge “are 


Figure 2: Running the weather/traffic view application. 
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you there?” events that the Application 
Manager sends to detect hung threads, its 
windows must deal with changes to the 
window stack, and it must be ready to 
give up resources if they start to run low. 
This app acknowledges the mandatory 
events, but to keep it simple, does not 
cope with things like another application 
changing the color format. 

After the /nit function is called, Appli- 
cation Manager calls the main function. 
The app now has its own thread, but can- 
not draw to the screen or even create win- 
dows until it gains focus. All of the stages 
of coming on-line are handled by send- 
ing the main function events. main ends 
up being a loop that repeatedly gets an 
event and dispatches it. Each event caus- 
es some work to be done, but there is lit- 
tle simple procedural code. 

The state machines keep track of whether 
a window is idle, reading an image from 
BFS, or waiting for a newer image to be 
placed on the server. Some of the state tran- 
sitions are due to user interface events and 
some are caused automatically by the up- 
date event or a read finishing. 

Once the app has come into focus and 
created and drawn all of its windows, it be- 
gins sending itself a periodic update event. 
This is mainly to know when to recheck 
the version of all BFS files that are waiting 
for a new one to appear. However, some 
state transitions are handled by switching 
to an intermediate state directly because of 
a user interface event and moving to the 
real state when the next update event is re- 
ceived. I could have created a separate 
thread for updates, but using a periodic 
event demonstrates how to weave two 
threads into one using events. 

Stream manager functions can be called 
asynchronously so they return immediate- 
ly and deliver events when the operation 
completes or fails, but the update event 
had already made things complicated and 
I wanted to demonstrate how easy it is to 
launch additional threads from an app. A 
function launched as a new thread must 
have a particular prototype, but beyond 
that it can do anything it wants. It can be 
pending on events from a queue or just 
run as a loop. When the work is done, the 
thread function simply returns and the 
thread context is destroyed. 


Conclusion 

If you are familiar with UNIX/X, Windows, 
or Macintosh, the PTVOS should seem fa- 
miliar with its standard C library, TCP/IP net- 
working, and a simple drawing library. How- 
ever, things start getting interesting when 
you write television-centric applications with 
overlay over video, cable-specific network- 
ing, and a nonkeyboard user interface. 
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The BeRays 





Ray Lracer 





Making it easy to plug 
in new rendering ideas 





Regan Russel 


eOS, an operating system from Be 
Inc. (http://www.be.com/), was de- 
signed for digital media applications 
on PCs and Internet appliances. Con- 
sequently, BeOS supports a 64-bit jour- 
naling filesystem, real-time multimedia 
streaming, pervasive multithreading, and 
multiprocessor support. 

There are two types of Be applica- 
tions —GUI applications and console ap- 
plications. A console application can be 
written in C and is comparable to UNIX 
programming. In fact, many tools are GNU 
based. The default shell is the BeShell, a 
slightly modified Bourne Again Shell 
(BASH), popular with users of FreeBSD, 
Linux, Solaris, and the like. The open- 
source versions of Lex and YACC—Flex 
and Bison— are also available. Addition- 
ally, for the Intel version, NASM (assem- 
bler with MMX support) is available. 

The Be GUI API, on the other hand, is 
C++ based. Unlike Windows (and XID, 
which lets you write apps in C using the 
raw Win32 API without using C++ and 
MFC (or Motif applications without a C++ 
Wrapper), you cannot write GUI-based 
BeOS applications in C. To do anything 
interesting, applications must create ob- 
jects derived from the classes BApplica- 
tion, BWindow, and BView in C++. 

In this article, I present a GUI-based Be 
ray-tracer application called “BeRays” — 
an object-oriented application (written in 
C++) that makes it easy to plug in new 
rendering ideas. Most public source ray 
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tracers are written in C and simulate in- 
heritance in case statements or by imple- 
menting their own virtual function tables 
(commonly known as “vtbls”) by manip- 
ulating function pointers. In BeRays, I use 
flex and bison to generate the parser for 
the input language.The source code and 
related files for the BeOS implementation 
of BeRays is available electronically; see 
“Resource Center,” page 5. Although I’ve 
tested BeRays on both Intel BeOS 4.5 and 
PowerPC BeOS 4.0, I’m only providing 
the Intel version with Intel project file. To 
build a PowerPC version, you simply cre- 
ate a new project and add all the required 
source files and libraries. 





BeOS Kits 

The keys to developing BeOS applications 
are kits (classes and interfaces to classes 
that provide support) and the Smalltalk 
Model View Controller (MVC) concept. 
There are a number of kits, including: 


e The Application kit. The most impor- 
tant kit for beginning BeOS program- 
ming is the Application kit. Like most 
C++ programs, a BeOS application 
starts in main(). The main function is 
usually used to create an application 
object and Run() it; see Listing One. 
When using a derived class version of 
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Listing One, the subclassed application 

object constructor usually constructs a 

window and view, then attaches the 

view to the window and the window 
to the application. The MIME encoded 
string is a signature used by the con- 
structor (BApplication::BApplication(). 

There is one string of this type for each 

application. The operating-system 

framework uses the signature string in 
the same manner that MIME is used in 
browsers. It knows to run applications. 

Windows COM experts might consid- 

er a Be application’s signature similar 

to a GUID in that it distinguishes com- 
ponents, or in this case applications, 
from one another. 

Additionally, the Application kit in- 
cludes BMessage, which provides func- 
tionality for application objects and is 
the basis for event handling. Any class 
derived from BHandler or Blooper can 
handle BMessages. In addition to nor- 
mal events, such as a window being re- 
sized, messages can be sent from one 
application to another. This is called 
“scripting” and supports chaining graph- 
ical applications together in the same 
manner that shell commands are linked. 
The Storage kit handles file I/O, direc- 
tories, MIME types, and filesystems. 
The Interface kit handles all GUI work 
and contains drawing, controls, view, 
and window classes. 

The Kernel kit facilitates low-level access 

to kernel features such as threads, teams 

(a collaborative collection of threads), 

semaphores, and linker images. 

The Media kit provides an interface to 

sound files and digital-to-analog (and 

back) conversion support. 

e The Midi kit supports Midi sampling and 
synthesizer files. 

e The OpenGL kit provides an interface 
to OpenGL support of GLScreens and 
GLViews. 

e The Device kit handles joystick, the 
three-fuse geek port, and serial I/O. 

e The Game kit enables direct window 
and direct screen access for games. 
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New! 


Together/J’s EJBs, GoF patterns, and simultaneous round-trip engineering—what Home 


Depot calls “the backbone of all our Java development’ 
Download the free TJ Whiteboard and experience WJ for yourself 
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Select a class, choose EJB™ or a GoF pattern, and TJ auto-generates 
lit for you — in Java or C++ (or both. in Toaether/Enterprise) 











UML 1.3 (nine kinds of diagrams) including r reverse engineering of 
methods into sequence diagrams. Plus HTML and RTF documenta- 
tion—invoke it as part of your daily build, deliver always-up-to-date doc 
to your boss 





Download TJ Whiteboard from www. togethersoft.com/dd/ “We discovered in our evaluation that other modeling tools only get used during 
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forced to wonder why anyone would like to work in any other way! | do think that that sia developers will use from start to finish. Why? It's the only eae that 
the strength of the product lies in its interoperability — it’s a ‘complete solution’, delivers simultaneous (rather than batch-mode) round-trip engineering.” — Curtis 
not just a modeling tool.” — Jon Collins, Senior Analyst, Bloor Research Chambers, The Home Depot 
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video discussion group of all our users to discuss the use case designs that we 
posted... All with Together/J. Your product does everything we need to do now and 
more...” — Steve McConnell, The Shaw Group 
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tools that merely support the Java-language syntax... Together/J 3’s support for 
key UML 1.3 diagrams, patterns, and Java-specific language features makes it a 
valuable addition to your Java development toolset.” — Jolt Award ‘99 


“| really like Together/J. No other vender has even come close to making such an 
intuitive and convenient diagramming interface.” —David Marine, Sr. Web 
Applications Developer, NetSolve 
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(continued from page 62) 

e The Network kit supports sockets and 
network services. 

e The Translation kit enables the devel- 
opment of filters for different file types; 
for example, PNG and BMP. 

e The Mail kit is a foundation of e-mail 
agent and delivery services. 

e The Support kit contains miscellaneous 
functionality including void pointer lists 
(BList) and profiler watchdog support. 


Kits talk to servers. Servers are daemon 
processes that manage things. One such 
server is the application server, which in- 
cludes a psycho-killer thread that goes 
around cleaning up processes that have 
gone wild, such as UNIX zombie pro- 
cesses. Kits also require a library and pos- 
sibly a header file to be included. For ex- 
ample, the OpenGL kit needs libGL.so and 
be/opengl/GLView.h. 

Kits are not only threadsafe but spawn 
threads themselves. Creating a window 
(from the Interface kit and managed by 
the application server) creates a separate 
thread to manage it. A group of threads 
are a team and an application is usually 
a team of a few threads. 

Many objects in kits take responsibility 
for their subobjects. For example, a BMenu 
will delete each BMenultem in its de- 
structor. It is often valid to allocate vari- 
ables without keeping track of them and 
not expect memory leaks. BeRays con- 
tains a few new statements without match- 
ing deletes. The general case still remains 
that if the OS allocates it, the OS frees it. 
If the application allocates it, then the ap- 
plication should free it if the OS will not. 
BList, for example, will not delete attached 
objects, but will delete its list nodes. 

Messages are generated from events. 
Events can have their handler specified at 
creation time. For example, the BFilePan- 
el in BeRay’s ray tracer directs its messages 
to the parent window. The messages the 
parent receives depend on the user choic- 
es when the dialog is shown. Some events 
are handled by member functions for ex- 
ample BWindow::FrameMoved( ). Threads 
do not receive messages. Message han- 
dlers receive messages. 


The BeRays Ray-Tracer Application 
The major components of the BeRays ray 
tracer I present here are the user inter- 
face, rendering engine, and rendering- 
engine support. The main function fires 
off the user interface components, and 
the rendering engine is then constructed 
and rendered (via the Go menu item, 
handled by the MessageReceived mem- 
ber function). 

The application starts in main and cre- 
ates a new RaysApp application, Runs the 
application, and then deletes the applica- 
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tion when the Run returns. The RaysApp’s 
(subclassed from BeApplication) construc- 
tor creates a RaysWnd (subclassed from 
BWindow). The window handles messages 
and the constructor creates a view. The 
RaysView constructor creates BMenuliems 
and a BMenu and attaches the menu items 
to the menu and the menu to the view. 
The message is directed to the window 
message handler when the menu item is 
constructed. 

The Go member is called from the win- 
dow’s message-handling function. The 
RaysWnd MessageReceived member is 
called when there is a MOPTION_RENDER 
message generated from the menu item. 
The menu item is created in the window’s 
constructor with BMenultem(Render', new 
BMessage(MOPTION_RENDER ) );. 

Another message the windows message 
handler deals with is the file open menu 
message. The file is read and input is 
passed to the flex scanner and bison gram- 
mar parser. The parser calls the Multi- 
Render member to create new objects. List- 
ing Two is the BNF grammar for the input 
language. 

The general approach I take to ray trac- 
ing is not new and is well documented by 
Glasner, Foley and van Dam, Heckbert, 
Watt, Watt and Watt, and others, includ- 
ing DDJ (see, for instance, “RAY: A Ray- 
Tracing Program in C++,” by Alain Man- 
gen, July 1994; and “Ray Tracing,” Daniel 
Lyke, September 1990). It is fairly well un- 
derstood that backward ray tracing starts 
at an eye or camera position and rays are 
traced through a view plane toward a 
scene full of objects. If the ray intersects 
an object, other rays are traced from the 
surface of that object for shadows, shad- 
ing, and reflection. If the ray does not in- 
tersect with an object, a background col- 
or is set. Forward ray tracing starts at a 
light source. Backward tracing is also 
known as “persistence of vision.” 

Because a photon is its own antiparti- 
cle, a ray going forward in time from a 
light source to a light receptor can be con- 
sidered the same as its antiparticle (itself) 
going backwards in time from the receiv- 
er to the transmitter. This has the added 
bonus that rays emitted from the source 
but never reach the receiver are not con- 
sidered. This is a significant speed im- 
provement. For a photon travelling at the 
speed of light, time has no meaning. 

For all rows and columns of the image, 
the Go member function constructs a ray 
starting from the viewpoint and starts a 
(potentially recursive) Trace through the 
scene, determining intersections of the ray 
with scene objects. The result of the Trace 
is passed to the Shade routine, which de- 
termines a color (possibly by calling Trace) 
and passes it back up to Go where it is 
set. The RenderEngine also maintains a 
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list of scene objects and a list of light 
sources. 

The Trace iterates through a list (BZist) 
of scene objects and determines the rays 
intersection (or lack of) by using the scene 
objects intersection function. The Shade 
member uses the scene objects CalcNor- 
mal interface to determine the amount of 
shading. In fact the Trace and Shade mem- 
bers directly or indirectly use a lot of sup- 
port functions. 

SceneObject is a base class for any ob- 
jects that appear in a scene. SceneObject is 
used in Trace, Shade, and LightSource’s 
GetColour member for shadow determina- 
tion. Scene objects need to support three 
interfaces: Construction, Intersection, and 
Calculate Normal. Light source, sphere, and 
plane are types of scene objects. New ob- 
jects and variations of existing objects can 
be added at will. Scene objects store prop- 
erties as a surface object. 

Light source is a type of scene object and 
is used in the shade routine. It contains the 
light’s color and position and has interfaces 
for getting the color and ray intersection. 
The method for getting the color determines 
if there are objects between the light source 
and the object the traced ray has stuck. This 
is used in shadow determination and is 
called a “shadow feeler.” 

In addition to scene objects, the Multi- 
Render class uses a color class to represent 
colors such as the light to define a scene. 
The color class stores red, green, and blue 
triples. It is used in shading and specify- 
ing objects properties. Colors can be spec- 
ified in the range of 0 to 1.0 or in 0 to 255. 
They are automatically converted. 

Vectors are used extensively through the 
render engine to generate an image from 
the definition. The vector class is a mathe- 
matical entity for storing x, y, and z triples 
with operator overloading for vector oper- 
ations. The methods and operations include 
length, rotate, reverse rotate, min, max, ~ 
(normalization), “ (cross product), % (dot 
product), * / +, and -. A ray is two vectors 
from an origin to a direction. All vectors 
only contain a direction as they start from 
a common origin of (0,0,0). 

The generation of the image starts with 
the viewpoint. The viewpoint is an eye 
position or camera position. A viewpoint 
contains four vectors: up, right, location, 
and direction. Rays originate from the 
viewpoint and once constructed with 
MakeRay, the viewpoint is discarded. 

With future development in mind, I de- 
fined an image class with a red, green, and 
blue channel bitmap so that per channel 
image processing could be done later. 

To build BeRays, you start at the com- 
mand line and compile the lex (flex) 
and yacc (bison) specifications into C 
sources with the make command and a 
standard Makefile. The project can then 
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be compiled and linked with the Build 
menu item from the IDE. 

When running BeRays, you are pre- 
sented with a window, blank white view, 
and menu bar. The menus include File, 
Go, and Help. The file menu consists of 
Open scene file and Quit. The Go menu 
contains the item Render and the Help 
menu contains About. The scene file is 
created with a text editor like vi. The pro- 
gram on start up does not contain a scene, 
so a default scene is used. 


Conclusion 

BeRays is simple and can be extended in 
a multitude of ways. Different illumina- 
tion methods could be tried, as well as 


adding bounding volumes, BSP trees, and 
octrees. One major problem to solve is 
spatial aliasing or jaggies. Antialiasing tech- 
niques (such as stochastic ray tracing) 
could also be applied. Image processing 
could be integrated. Other types of ob- 
jects can be added: You need to create a 
derived class for the new object and over- 
ride the Intersection and CalcNormal 
members and add lex and yacc descrip- 
tions for the new object and a function in 
the render engine to add the object passed 
in from the parser to the scene list. 
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e e . e 
Listing One Listing Two 
#include <Application.h> DIGIT ::= 0)1121314;5161718)9 
main () NUMBER: := DIGIT {DIGIT} 
{ SCENE: := OBJECT | SCENE OBJECT 
const int left = 10, right = 500, top = 10, bottom = 200; OBJ: := 


// Create application, window and view.. 


BApplication *TheApp = new BApplication( "application/x-vnd.ddj-sample" ); 
BWindow *TheWindow = new BWindow(BRect( left, top, 
B_TITLED_WINDOW, B_NOT_ZOOMABLE |; B_NOT_RESIZABLE ); 
100, 200, 200 ), 

B_WILL_DRAW ); 

show the window and run the application. 


"Rays" , 
BView *TheView = new BView( BRect( 120, 
"Rays", B_FOLLOW_ALL_SIDES, 
// Attach view to window, 
TheWindow->AddChild( TheView ); 
TheWindow->Show() ; 
TheApp->Run() ; 
// Clean up. 
delete TheView; 
delete TheWindow; 
delete TheApp; 


right, bottom ), 


BACKGROUND_OBJECT: := 
SPHERE_OBJECT: := 
COLOUR_REF CLOSE_BRACKET 
COLOUR_REF: := 
COLOUR_VAL: := 
RADIUS: := { NUMBER } 
PLANE_OBJ: := 
CLOSE_BRACKET 


Go Beyond 


For developers that need advanced 
visualizations, high-performance data 


analysis and development tools, IDL is your Sshidon, 
IDL®, the Interactive Data Language, is a powerful 


Stom 


4GL that delivers state of the art graphics, produces 
cross-platform applications and handles common, 
emerging and user-defined data formats. 


IDL 5.2 Development Feature Highlights 


V IDL Development Environment 


Y IDL GUIBuilder 
Y ActiveX Control 


Y OpenGL Accelerated 3D Graphics 

V Extensive Suite of Math & Statistics Routines 
Y Object-Oriented Graphics Technology 

Y Pre-built Visualization Applets 

V Full Set of GUI Components 

V Flexible Data I/O 


Combining robust code editing, profiling and debugging with 
an extensive suite of processing routines and leading 
visualizations, IDL is equal to any requirements. Once you see 
how easy IDL is for solving big visualization problems — you'll 
wonder why you waited so long. 
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Designing the legOS 


Multitasking 
Operating System 





An OS for the Hitachi 
HS microcontrolle 


Markus L. Noga 


indstorms is a system developed 
by Lego and the Massachusetts 
Institute of Technology (MIT) that 
lets you design, build, and pro- 
gram robots using standard Lego pieces, 
a handful of sensors, and a microcon- 
troller that is programmable via an in- 
frared link with a PC. More specifically, 
the Mindstorms Robotics Invention Sys- 
tem (see http://www.legomindstorms 
.com/) consists of more than 700 Lego 
pieces, two motors, gears, two touch sen- 
sors, a light sensor, the microcontroller, 
an infrared transmitter, and a PC cable 
to connect it to one of your COM ports. 
The packaged software currently sup- 
ports Windows 95/98/NT. 

Constructing Mindstorms robots in- 
volves combining normal Lego pieces 
with sensors (see Figure 1) that can han- 
dle data input and enable the robot to 
respond to external situations. You pro- 
gram the Robotic Command Explorer 
(RCX) computer (Figure 2) in a graphi- 
cal language called “RCX Code.” Finally, 
as Figure 3 illustrates, the code is then 








Markus is a student of computer science at 
the University of Karlsruhe, Germany. He 
can be contacted at noga@inrialpes.fr. 
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downloaded from your PC to the RCX 
via an infrared link. 

Although you typically program the 
RCX using RCX Code, you can also use 
SPIRIT.OCX (provided in the Mindstorms 
SDK) to control the unit using Visual Ba- 
sic, Visual C++, Delphi, or the like. Fig- 
ure 4 illustrates the RCX software sys- 
tem, which consists of a graphical 
programming environment, OCX control, 





and bytecode Virtual Machine (VM). Al- 
though powerful, each of these layers 
introduces certain limitations. The graph- 
ical environment limits variable usage to 
one counter, rendering it unusable for 
any but the simplest applications. The 
event-driven programming model is 
severely hampered by the lack of reen- 
trancy and task synchronization. And fi- 
nally, like the OCX control, the envi- 
ronment is Windows only. 
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The VM, on the other hand, provides 
32 global variables, a write-only datalog, 
eight concurrent tasks, and direct con- 
trol from a PC. Still, the VM also suffers 
from drawbacks, lacking, for instance, a 
stack, a real heap, and the concept of 
priorities. 


Alternative Software 
Because of these and other limitations 
in Mindstorms software, a number of 
freeware/shareware alternatives have be- 
come available, since Kekoa Proudfoot 
reverse engineered the bytecode in- 
struction set C(http://graphics.stanford 
.edu/~kekoa/). Dave Baum conceived of 
Not-Quite-C (NQC), a C dialect suitable 
for the bytecode VMs. This implies no 
local variables, compound expressions, 
reentrancy, arrays, or user-defined data 
types. Baum’s NQCC compiler, now in- 
tegrated in his RCX Command Center, 
runs on a wide variety of host platforms 
and is still the most widely used byte- 
code cross-compiler. Other tools include 
RCX Creator (Barry Ruffner), BotCode 
(Marty Balash), and Bot-Kit (based on 
Dolphin Smalltalk), along with tools and 
techniques for programming the RCX 
with VBScript or JavaScript. 
Programming environments centering 
around NQCC and other compilers tar- 
geting the RCX VM followed, and high- 
level wrappers for the OCX control pro- 
vided by Lego abound. However, all these 
approaches share the limitations imposed 
by the VM. Frustrated with these short- 
comings and the unavailability of so many 
crucial features, I launched the legOS pro- 
ject as an alternative to the RCX firmware. 
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(continued from page 606) 

My goal was to design a small embed- 
ded operating system, implemented in 
standardized languages and native as- 
sembler instead of exotic dialects and byte- 
codes. In other words, I wanted to tear 
down the padded VM environment while 
providing similar or superior system ser- 
vices. legOS offers a number of features, 
including preemptive multitasking, energy 
saving, dynamic memory management, 
POSIX semaphores, native access to dis- 
play, buttons, infrared (IR) communication, 
motors and sensors, and a new kernel and 
library. legOS can be programmed with stan- 
dard C or assembler. It is available from 
DDJ (see “Resource Center,” page 5) and at 
http://www.multimania .com/legos/. 


The Real Machine 

The heart of the Mindstorms RCX is a 16- 
MHz Hitachi H8/3292 series microcon- 
troller. This processor line sports an 8- 
channel 10-bit A/D converter, serial 
communications interface, several timers, 
and multiple digital I/O ports among its 
on-chip modules. The version inside the 
Mindstorms offers 16 KB of ROM and 32 
KB of RAM. Fortunately, the H8 instruc- 
tion set is a supported target platform for 
several flavors of the freely available GNU 
C compiler. I used binutils-2.9.1 and gcc- 


2.8.1 under RedHat Linux 5.1. Pre-1.1.1. 
egcs and other gcc versions might need 
minor patches to cross-compile correctly, 
so stick with gcc-2.8.1 if possible. 

These tools are available from any num- 
ber of sources, including the GNU ftp site 
mirrors and most Linux distributions. Their 
installation on a UNIX-like platform is a 
near-classic case of “make ; make install;” 
see Example 1. Installation under Win- 
dows is equally straightforward with the 
aid of cygwin32, a free UNIX compatibil- 
ity package by Cygnus Support (http:// 
www.cygnus.org/). 


Design Decisions 
I chose to comply with standard UNIX 
interfaces wherever possible. Most C pro- 





Figure 1: Lego touch sensor. 


grammers are familiar with puts), sleep(), 
malloc(), and exec(), and I wanted to 
ease their transition. However, this is also 
a question of personal taste, because I 
dislike the abbreviated, technical naming 
conventions in other embedded operat- 
ing systems that might have served as 
role models. 

Outside the VM sandbox, there is no 
memory protection whatsoever; like many 
embedded systems, the H8 simply lacks 
the concept. Malformed or malicious user 
applications can always overwrite kernel 





Figure 2: The RCX computer. 
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Dr. Dobb’s Essential Books on 
Numerics and Numerical 
Programming CD-ROM 


This comprehensive reference 

tool will give you: 

¢ Detailed introduction to numerics, 
numerical analysis, and their 
applications 


¢ Solutions to numerical problems 
based on limited memory storage space for data and | 
applications 


¢ Thorough up-to-date treatment of the behavior of 
numerical algorithms, precision arithmetic, combining 
algorithmic derivations, and rounding analysis 


¢ Multigrid methods for solving partial differential equations 
¢ Spectral analysis on unevenly sampled data 

¢ Calculation of numerical derivatives 

¢ Write efficient, high-speed arithmetic routines, and more! 


Dr, Dobb's Price: $99.95 


| CALL: 800-992-0549 











or visit our website: 
co-ROM www.ddj.com/cdrom/ 
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(continued from page 68) 
code and data. Consequently, I sacrificed 
security for speed and simplicity. 

To minimize memory footprint, I want- 
ed to give programmers full and fine- 
grained control over which functionali- 
ty to include in the kernel. As in most 
embedded operating systems, this can 
easily be achieved with conditional de- 
fines.The full system’s memory require- 
ments are currently a third of the origi- 
nal firmware’s. 

The kernel was to be statically linked 
to user applications, forming a binary 
downloadable with standard firmware 
tools. This was the only initially feasible 
method, as an IR driver didn’t exist. Also, 
it emphasizes independent operation of 
the RCX. Dynamic linking of user appli- 
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cations against a kernel symbol table on 
a host PC will be a future option for task 
download via IR. 

I decided against a filesystem, since 
there’s no mass storage on the RCX. And 
although user-mode hardware access via 
special device files is an elegant method 
on systems with memory protection, the 
operating security gained on the H8 would 
be illusory. Currently, I like to view the 
hardware devices as OS-provided objects 
with a thin public interface mostly con- 
sisting of inline functions. Device files 
might be considered in the list of possi- 
ble future extensions for the sake of ele- 
gance. Then again, when did ioctl()s be- 
come elegant? 

By textbook definition, the three basic 
tasks of an operating systems are: 
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e Memory management. 
e Resource management. 
e Task management. 


Lacking a complete emulator for the RCX 
hardware, resource access was clearly the 
first milestone to tackle. Without I/O, it’s 
difficult to see if your programs work. As 
tasks need stacks, memory management 
became the second milestone and task 
management the third. Thus, the course 
of development was set. 


Resource Management 
Due to space constraints, I'll only address 
the console driver— that is, the LCD dis- 
play and button control — in this article. Al- 
though most of the other drivers are fairly 
straightforward derivations of the H8 hard- 
ware manual (see http://www.multimania 
.com/legos/), nearly every single one con- 
tains RCX-specific twists. For example, the 
IR driver needs to enable carrier frequency 
generation exclusively when sending, and 
the A/D driver has to provide pulsed al- 
imentation for active sensors such as light 
and rotation. 

The starting point for the console driver 
involved the ROM functions to display sym- 
bols and numbers on the LCD and refresh 
the display and query buttons. Their ad- 
dresses and some of their parameters had 
been located by Kekoa, who used these 
functions in his first assembler firmware. 

In an early version of legOS, I relied ex- 
clusively upon ROM to address the LCD. 
Examination of the ROM disassembly nar- 
rowed the region of possible control 
codes. I then wrote experimental programs 
that stepped through these ranges and 
noted the display changes affected. 

ROM functionality includes 4- and 1- 
digit decimal number display with an op- 
tional selectable decimal point. This is how 
ROM and firmware display sensor values, 
program numbers, and time. There are also 
codes to set and unset individual graphics 
symbols such as the IR transfer indicator. 
legOS makes these capabilities accessible 
via symbolic constants and inline wrappers 
in rom/Icd.h. 

(continued on page 73) 






Figure 3: Downloading code from a | 
PC to the RCX via an infrared link. 
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(continued from page 70) 

In-depth analysis of the ROM disas- 
sembly revealed further potential for en- 
hancements. All ROM display functions 
employ the memory area Oxef43-Oxef4b 
as a bit field indicating segment states. 
From this buffer, the display is updated 
by the /cd_refresh() function. Recogniz- 
ing the rare possibility to improve both 
footprint and execution speed, I added 
direct-lcd.h to access the data area directly, 
bypassing ROM for all but display refresh 
(see Example 2 for a comparison). Grad- 
ually bypassing ROM completely is a gen- 
eral pattern in legOS development. 

Notice how arguments to ROM func- 
tions need to be recopied. ROM calling 
conventions differ from gcc’s —16-bit ar- 
guments and return values are passed in 
ro, not 70. Unfortunately, gcc’s excellent 
support for assembler instructions with C 
operands only offers constraints for reg- 
ister classes, not individual registers (see 
Example 3 for an example of how button 
states are read). gcc also treats 70..r3 as 
clobbered by function calls, although ROM 
preserves them. Trivial changes to the H8 
machine definition file would have ac- 
complished seamless compatibility with 
ROM, but I chose not to provide compil- 
er patches for every gcc version or bina- 
ries for every platform. 


Figure 4: RCX software layers. 
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Total system memory of slightly less than 
32 KB and an overall design goal of small 
footprint ruled out page-oriented alloca- 
tion schemes immediately; on average, 
they waste half a page per allocated block. 
Also, the smaller the blocks, the bigger 
their relative administrative overhead. 
This left continuous allocation. Of the 
possible variants, I decided to use a 
straightforward scheme (see Table 1) that 
is optimal in terms of space allocation and 
linear with a low constant in the time do- 
main. Why space optimal? For each allo- 
cated block, you need to store its length 
somewhere if you ever intend to free it. 
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With a 32-KB address space and a pro- 
cessor in need of word alignment for all 
but character data, this translates to 14 bits. 
Furthermore, you need to track a block’s 
status with a minimum of three cate- 
gories — free, allocated, and reserved (for 
memory-mapped I/O areas). This would 
fit in the remaining two bits. 

For multitasking systems, however, it is 
desirable to track block owners, if only to 
clean up after task death. With the con- 
straint of word alignment, this translates 
to another two bytes of administrative data. 
Free and reserved blocks are efficiently 





Table 1: Memory allocation scheme. 





Table 2: Hardware memory layout. 





implemented as special owner PIDs 
Ox0000 and Oxffff, respectively. 

To minimize fragmentation, freed blocks 
must be united with neighboring free 
blocks. This, however, means traversing 
the block list from the beginning to locate 
the predecessor. malloc() performs this 
traversal in any case, so unification was 
integrated there, leaving free() an atomic 
O(1) operation. Since the priority sched- 
uler has to free process data and stack ar- 
eas, this property is handy. 

The memory-managed area starts im- 
mediately after the last byte of static data. 
Fortunately, the GNU linker provides a 
method of inserting symbols at precise lo- 
cations in the memory map (see Example 
4). This spares you from clumsy double 
compilation or compile-and-patch ap- 
proaches. As a final common optimiza- 
tion, a pointer to the first free block is 
maintained. 


Task Management 
My next goal was preemptive multi- 
tasking. The subgoals involved in chang- 





Example 1: Cross-compiler setup. (a) Compiling and installing binutils; 
(b) compiling and installing gcc. 





Example 2: Direct LCD manipulation versus ROM calls. 
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ing to a new task context can be clear- 
ly separated in a task switcher (which 
performs the actual context switch) and 
task scheduler (which decides where to 
spend the next timeslice). The latter should 
ideally be totally platform independent, 
whereas the former is machine-specific 
by definition. 

To keep the scheduler interface lean 
and mean, I saved all context information 
on the stack frame. The only argument to 
the scheduler is the current task’s stack 
address, and the only return value is the 
next task’s. Consequently, the switcher is 
ignorant of process data structures. Start- 
ing a new task became the question of 
assembling a stack from which the switch- 
er would return to the desired start ad- 
dress and having the scheduler return this 
stack address. 

Initially, I selected a barebones circular 
scheduling scheme for maximum stabili- 
ty. The harder part was getting the switch- 
er to work. I started with a cooperative 
approach, forcing tasks to yield explicit- 
ly. This gave me a semistable environment 
wherein I could control the exact moment 
a task switch would occur. 

Trying to stick with UNIX standards, I 
examined various methods of task start- 
up and shutdown. A fork seemed infeasi- 
ble, as it requires a full copy of both data 
and stack segments. It is also overkill for 
most embedded applications. Finally, I ex- 
tended the exec family with execi(). This 
function inserts a new process into the 
scheduler and allocates stack space for it. 
execi() is nonstandard in that it doesn’t 
replace the old process, but simply adds 
a new one. In more orderly fashion, exit() 
ends a process’ life, and Rill() allows it to 
prey on others. 

After this system proved reasonably 
stable, I plunged into the joys of IRQ 
handling. As Table 2 shows, Hitachi 
placed the H8’s IRQ vector table at 0x0000 
as usual— a ROM address. However, in 
accord with the scheme of downloadable 
firmware, Lego’s engineers displayed 
some excellent software engineering. They 
created a secondary IRQ table in RAM 
and a primary ROM wrapper for every 
IRQ (see Example 5). The secondary IRQ 
handlers can be easily overridden by new 
firmware. Replacement handlers have to 
return with a normal “subroutine return” 
instruction, and they can rely on 76 be- 
ing saved and restored by the wrapper. 

The H8/3929 series provides one 16- 
bit and two 8-bit on-chip timers, each 
with several associated interrupts for 
compare-to-constants and overflow 
events. Fractions of the system clock are 
available as internal clock frequencies, 
as well as various external trigger modes. 
The 16-bit timer can generate two PCM 
waveforms, the 8-bit timers provide one 
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(continued from page 74) 
each. Additionally, an on-chip watchdog 
timer is available. 

Experimentation with the timers quick- 
ly indicated that the first 8-bit timer’s out- 
put was wired to the speaker, ruling it out 
for task switching. As the second 8-bit 
timer appeared to be safe, I employed it 
for my first scheduler. Later, I discovered 
it was generating the IR carrier frequen- 
cy, so I moved the task switcher to the 
16-bit timer. 

It was somewhat disappointing to no- 
tice that some IRQ handlers may crash 
the system. For IRQs associated with cer- 
tain status bit changes, the handlers have 
to reset the status bit before returning or 
the system will freeze. You should al- 
ways mask out IRQs instead of provid- 
ing empty handlers. The otherwise com- 
prehensive, freely available hardware 
manual from Hitachi (ink on http:// 
www.multimania .com/legos/) fails to ad- 
dress this fact. 

The current scheduling algorithm is 
priority based. Per default, only pairwise 
different priorities are admitted, as this 
allows for considerably faster schedul- 
ing. Support for equal priority levels by 
priority rotation within a level can be 
compiled in. 
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Example 5: Disassembly of a ROM IRQ wrapper. 











Processes may wait for arbitrary event 
functions to become nonzero — this is 
how sleep() is, and select() may be im- 
plemented. To conserve battery power, 
I introduced a special idle task of low- 
est priority. It switches the processor to 
sleep mode until the next interrupt oc- 
curs in case all regular processes are 
waiting for events. 


Additional Features 

The combination of memory and task 
management prompted a means of syn- 
chronizing access to critical sections. 
POSIX semaphores are an accepted so- 
lution to this problem, and the POSIX 
standardized interface was easy to im- 
plement. 

Writing an IR driver was frustrating un- 
til I had a flash of inspiration. No mat- 
ter what settings I tried for the H8’s on- 
chip serial module, the PC wouldn’t 
receive anything. Then I realized the car- 
rier frequency had to be generated on- 
chip. The IR driver supports half-duplex 
operation at arbitrary baud rates and col- 
lision detection. Hardware constraints 
limit usable transmission speeds to 4800 
baud, but this allows over four-times 
faster communication than the original 
firmware for limited distances. For the 


Dr. Dobb’s Journal, November 1999 


near future, there is talk of writing pack- 
et networking code. 

About that time, another programmer 
discovered the motor driver I/O address 
to be Oxf000. That same day, I wrote a 
first PCM driver. A later discussion on 
the Mindstorms mailing list led to an ad- 
vanced PCM driver derived from the Bre- 
senham line- drawing algorithm. It offers 
256 speed levels instead of the eight 
available with the standard firmware. 

Support for basic resistive sensors (that 
is, the touch and light sensors packaged 
with the Mindstorms set) was straight- 
forward to add with code taken almost 
literally from the H8/3292 hardware man- 
ual. Guessing conservatively, a 10-KHz 
sampling rate can be achieved. 

Making the light sensor measure re- 
flectivity instead of just ambient light was 
largely trial-and-error. Power for the LED 
had to come from some specific output 
pin, and there were only a handful to try. 
As the cable to the sensor has only two 
wires, alimentation and A/D conversion 
are mutually exclusive. They have to be 
synchronized. Rotation sensors addition- 
ally require a state machine to decode. 

Examinations of the RCX hardware in- 
dicate that two additional A/D pins might 
be usable. Should this be confirmed, leg- 
OS will support two additional sensors. 


Conclusion 

There’s plenty of room for improvement 
in legOS. Apart from C++ support, device 
files, and enhanced stability, most items 
on the wish list depend on IR packet net- 
working. 

A shell task could provide a clean 
method of loading applications via IR as 
previously outlined, saving application de- 
velopers from downloading the kernel ev- 
ery time. Together with enhanced IR 
speed, this could potentially divide pro- 
gram download times by four or more. 

A gateway application on the host com- 
puter could provide TCP/IP connectivity, 
enabling services such as local and remote 
joystick control and status display. Com- 
mercial products such as EMIT 3.0 Chttp:// 
www.emit.com/), available with an H8 eval- 
uation board, might serve as an inspiration. 

Packet networking would also vastly 
extend the possibilities of inter-RCX com- 
munication, facilitating the development 
of systems of interacting agents. 

legOS already fills a technically inter- 
esting niche for development targeting the 
Lego Mindstorms. With some of the afore- 
mentioned extensions, it might become a 
feasible platform for courses in introduc- 
tory robotics, mobile agent systems, and 
certain forms of rapid prototyping. 
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JBED: Java for 
Real-Time Systems 





A JVM and an RTOS 





Jorgen Tryggvesson, Torbjorn 
Mattsson, and Hansruedi Heeb 


bed is a small, fast Java Virtual Machine 
(JVM) for embedded real-time systems. 
It also includes a complete real-time op- 
erating system (RTOS). In designing 
Jbed, our first goal at Esmertec (http://www 
.esmertec.com/) was to make a seamless 
combination of an RTOS and JVM without 
any glue code in between. Our second de- 
sign goal was not to rely on an interpreter 
or just-in-time (JIT) compilation. Instead, 
Jbed uses either a precompiled native im- 
age, or a built-in compiler that translates 
Java bytecode into native code at load time. 
(This technique is usually referred to as 
“flash compilation,” but since someone 
trademarked that term, we call it “TBCC,” 
short for “target bytecode compilation.”) 
Because Jbed always runs natively, it is 
a very fast Java system. That it is also a 
small system may be surprising, but leav- 
ing out the interpreter and not relying on 
techniques such as JIT saves memory. Ad- 
ditional memory saving is achieved by 
writing everything in Java— even the de- 
vice drivers (to write device drivers in Java, 
Jbed provides a built-in class with meth- 
ods for direct memory access). Of course, 
you can link to legacy code written in C, 
but if you refrain from that, only the Java 
memory management has to be loaded, 
and a lot of overhead is avoided. A com- 
plete Jbed system with TBCC is only about 
250 KB and uses about the same amount 
of RAM to run. A system with TCP/IP and 


Jorgen is development engineer at Mecel AB. 
He can be reached at jorgen.tryggvesson@ 
mecel.se. Torbjorn is a software consul- 
tant working at Mecel AB and can be con- 
tacted at info@mecel.se. Hansruedi is CEO 
of Esmertec Inc., and can be contacted at 
hh@esmertec.com. 
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a web server is about 128 KB. Minimal 
precompiled systems without the TBCC 
can be smaller than 10 KB. These num- 
bers include both the RTOS and JVM. 

TBCC technology has another benefit. 
Code loaded dynamically into the running 
system is compiled immediately, then exe- 
cuted at full native speed. This makes Jbed 
ideal for applications where interpreters are 
too slow, but where hot-loads of patches 
or new applications are necessary (as with 
some Jini applications, for instance). 

A final design goal was to make Jbed 
suitable for hard real-time applications. 
Hard real-time usually means that no out- 
side event must ever be handled too late. 
More precisely, each event has a given 


deadline and missing any such deadline 
results in an error condition (soft real-time, 
on the other hand, allows for occasional 
deadline misses). Real-time applications 
are more challenging under a Java-based 
operating system than under other sys- 
tems because Java uses garbage collec- 
tion. The garbage collector used in Sun’s 
original JVM cannot be interrupted and is 
therefore unsuitable for real-time appli- 
cations. Instead, an incremental garbage 
collection algorithm has to be used that 
can be interrupted anytime. 

Jbed is a clean-room design and con- 
tains no Sun sources. It comes with its 
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own integrated development environment 
(IDE), or can be used with Symantec’s 
Visual Café, IBM’s VisualAge, and the like. 
Jbed runs on Motorola 68xxx and Power- 
PC processors, with Intel, ARM, and MIPS 
processors support coming soon. 


Hard Real Time 
Contrary to popular notions, hard real- 


time problems with extensive hardware 
interaction can be solved entirely in Java. 
To illustrate this, we present a hard real- 
time example that is simple and main- 
tainable, while still meeting hard real-time 
design requirements. In doing so, we 
won't link C code for the device drivers, 
but instead use the hardware interface li- 
braries that come with Jbed. This not only 
makes memory management easier, but 
has advantages in terms of documentation 
and code maintenance. Because the prob- 
lem is hard real time, Java systems that 
are interpreted or rely on techniques such 
as JIT compilation can’t be used. 

The problem at hand involves a robot 
with two moving axles, both under control 
of the system; see Figure 1. The robot has 
an attached laser-pointer that is used to dis- 
play Lissajous figures on a screen (a Lis- 
sajous figure is the superposition of the two 
sine functions x()=Asin(ct) and y= 
Bsin(dtt+e) plotted in the x-y-plane; see Fig- 
ure 2. This example is hard real time be- 
cause the movements of the robot have to 
be monitored and adjusted in very short 
time intervals (milliseconds) to achieve the 
desired accuracy. The accuracy of the sys- 
tem can be easily tested by displaying a Lis- 
sajous figure where the two sines have the 
same frequency and no phase difference 
because the resulting figure must be a line. 
The slightest inaccuracy of the timing 
makes it into an ellipse instead. Finally, the 
robot system will be implemented as a 
time-triggered (deadline-driven) system. 


Event-Triggered versus 

Time-Triggered Systems 

In event-triggered systems (based on in- 
terrupts) that use multiple sensors (heat sen- 
sors, Oil-pressure sensors, sensors triggered 
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(continued from page 78) 

once per rotation of a moving part, etc.), 
each trigger is an interrupt handled by an 
interrupt routine. The interrupt routines are 
short, and as much work as possible is del- 
egated to nonreal-time threads. (Here, we 
use the term “thread” exclusively for Java 
style, nonreal-time threads, and we use 
“task” for real-time tasks.) Each interrupt 
routine has a priority. Routines with higher 
priority will interrupt routines with lower 
priority. 

In such a system, priorities have to be 
assigned (possibly dynamically) such that 
each external event is serviced within its 
deadline. For instance, the interrupt rou- 
tine that must be called once per rotation 
of a moving part must have a priority that 
guarantees it is still called once per rota- 
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tion even under special circumstances 
(such as a malfunction of another part of 
the machine). If this is not guaranteed, a 
malfunction could spread quickly. In a 
complex system, this priority assignment 
can be very difficult to debug. 

In time-triggered (deadline- driven) sys- 
tems, on the other hand, all sensors are 
sampled (polled) at regular intervals. Each 
sensor causes the execution of a small 
piece of code, similar to an interrupt rou- 
tine. Again, as much of the calculation as 
possible is delegated to nonreal-time 
threads. Instead of priorities, the routines 
triggered by the sensors have deadlines 
that are usually known a priori and need 
not be assigned. For example, a rotating 
part that rotates a maximum of 1000 times 
per second has a natural deadline of 1 
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millisecond. The scheduling is based on 
which deadline is the most urgent (earli- 
est deadline gets first scheduling). 
Because the routines usually do not per- 
form complex calculations, it is possible to 
assign them a duration (that is, a maximum 
execution time). Again, the duration is 
known a priori and, therefore, the sched- 
uler can do a load-time analysis of the 
schedule of the system under all circum- 
stances. After all modules are loaded (ei- 
ther statically and one-time or dynamical- 
ly), Jbed guarantees that the system can be 
scheduled under all circumstances if the 
durations of the real-time tasks were giv- 
en correctly. If the system cannot be sched- 
uled, it does not even start executing. The 
requirement to have a correct estimate of 
the duration of the real-time tasks is not a 
serious problem in practice, because well- 
designed tasks are simple and the estimates 
can be on the safe side (and the actual du- 
rations are given in the Jbed debugger). In 
our robot example, it is guaranteed that 
the service routine for all moving parts is 
executed in time under all circumstances. 
How does the responsiveness of event- 
triggered systems compare to their time- 


triggered counterparts? Under small loads 


(few external events), event-triggered sys- 
tems are more responsive because each 
event is serviced immediately (unless a 
higher priority event comes in between). 
Time-triggered systems have a response 
time given by the sampling period. Under 
high loads, however, time-triggered sys- 
tems excel because they have much less 
overhead than event-triggered systems. 
Time-triggered systems have another 
advantage. If the machine in our robot ex- 
ample is extended by another rotating part, 
it is easier to fit the new control software 
into the system if it is time triggered. The 
service routine of the new part has a 
known deadline and duration, and can 
therefore be added to the system without 
changing the existing service routines. If 
the system can still be scheduled, no fur- 
ther work has to be done (if it is not, Jbed 
detects this at load time and the system 
will not start up). Event-triggered systems, 





Figure 1: The robot built by the 
Institute of Robotics, ETH, Ziirich. 
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(continued from page 8O) 

however, will likely require many changes. 
Time-triggered systems, thus, pave the road 
for the use of software components (like 
JavaBeans) in real-time systems. 


Java Library for Real Time 
Jbed does not make any changes to the 
Java language. To support real-time em- 
bedded systems, Jbed adds library class- 
es— some of which are built in (to access 
hardware directly from Java, for instance), 
while others are regular libraries written 
in Java. The most important such library 
is the task library, com.jbed.tasks. 

The task library defines a class Task, 
which is a subclass of java.lang.thread. 
Real-time tasks in Jbed therefore behave 





Figure 2: A Lissajous curve. 
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exactly like Java threads, except that they 
give additional guarantees concerning their 
real-time behavior. As with Java threads, 
Task objects control the behavior of a con- 
current activity; they contain all the book- 
keeping information necessary to do this. 
You implement the activity itself by writing 
a class that implements the interface 
java.lang.Runnable — again, just as with 
regular Java. The run method of this inter- 
face contains code that makes up the con- 
current activity. As in Java, an alternative to 
this approach is to subclass the 7ask class 
and implement user code in the 7m method 
of the class. In typical nonreal-time pro- 
grams, a thread’s 7m method contains an 
endless loop. This is different for real-time 
tasks: A Task should be restarted by Jbed 
so that it can automatically start the next it- 
eration at the right time. 

The Jbed scheduler uses timeslices to as- 
sign computation time to tasks. A timeslice 
is an implementation-dependent short pe- 
riod of time; one millisecond, for instance. 
The implementation’s timeslice is provided 
in the static variable com.jbed.tasks. Task.time- 
slice, see Listing One. The scheduler installs 
a timer interrupt to get control (after a time- 
slice has passed) by preempting the cur- 
rently running task. The scheduler then 
checks whether the previously running task 
has exceeded its admitted duration; if so, it 





is aborted. The scheduler determines which 
task has the closest deadline to meet, and 
lets it run for the next timeslice. Reschedul- 
ing may occur because of preemption after 
a timeslice, and also voluntarily when the 
running task stops or blocks (for instance, 
by sleeping or waiting for a signal). 

Jbed distinguishes between the six types 
of real-time tasks in Table 1. The type of 
Task is given in the constructor using the 
RealtimeEvent argument. 

When a task is created, its constructor 
initializes it if the scheduler’s admission test- 
ing was successful. This is the case if the 
scheduler can provide sufficient computa- 
tion time to the new task; that is, if other 
tasks have not yet reserved too much of 
the processor’s capacity. Otherwise, a 
com.jbed.tasks.AdmissionFailure exception 
is thrown. Listing Two is a typical exam- 
ple of how a task is created. The first state- 
ment creates an instance of one of the real- 
time event classes. Depending on the chosen 
event class, the real-time behavior of the task 
is different. Depending on the event class, 
you may need to use a tncatch statement 
for creating the event object. The second 
statement creates an instance of a runnable 
object (an instance of a class implementing 
the java.lang.Runnable interface). This ob- 
ject implements your application-specific be- 
havior. The try/catch statements create the 
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task object, using the event object and the 
runnable object as parameters, plus the du- 
ration, allowance, and deadline values spec- 
ified in microseconds. The last statement 
starts the task, which means that it is 
marked as ready to run. The scheduler then 
executes the task at the appropriate time. 
Two arguments to the 7ask constructor, 
duration and deadline, have been ex- 
plained already. The additional argument 
allowance requires further explanation: It 
is the time required for the handler of the 
DurationOverflow exception that is thrown 
if the task exceeds its duration. Without an 
allowance for this, the system might not 
have any time left to schedule it. Frequently, 
this situation is simply an error condition 
that requires no handling, and is removed 
during the debugging phase. Then the al- 
lowance can be set to zero. The task’s con- 
structor throws an IllegalArgumentException 
exception unless the following holds: 


duration > 0 
allowance >= 0 
deadline > duration + allowance 


If a system contains both Java threads 
and real-time tasks, Jbed schedules the 
threads whenever no task is running ac- 
cording to its priority (round-robin if sev- 
eral have the same priority). An example 
of such a thread is the garbage collector. 


The Robot Example 

Recall that the goal in our real-time robot 
example is to move two orthogonal robot 
axles such that their combined movement 
follows a Lissajous figure with given pa- 
rameters. The example has three real-time 
tasks and no user-defined Java threads. 
One of the three tasks, the WatchDog, is 
simply used to check if the software is 
alive and running. If it is not, the hard- 
ware shuts down. The other two are the 
Planner task and Control task. 

To reduce the memory footprint, the ex- 
ample avoids repeated dynamic memory 
allocation and the target bytecode compil- 
er (TBCC) is not used. Jbed is modular, and 
unused modules (such as the garbage col- 
lection or the TBCC in this case) are not 
loaded. Dynamic memory allocation is 
avoided by using static variables. Listing 
Three presents the definitions of some of 
these variables and definitions (the com- 
plete listings are available electronically; 
see “Resource Center,” page 5). 

The Planner task simply calculates the 
desired position of the two axles from the 
given parameters at the current time and 
then advances time. The sine method 
(from Jbed’s math library com.jbed.math 
.FMath) is used instead of the Java math 
library, which is far too large for embed- 
ded systems; see Listing Four. 
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The Control task has the more complex 
job of comparing the actual and the de- 
sired positions and calculating a reason- 
able movement for the current timeslice. 
The classes com.jbed.peripherals.Counter 
and com.jbed peripherals. AnalogActuator 
interface to the actual hardware and al- 
low mapping of, for example, an analog 
input reading from hardware to a given 
floating-point range. In our robot exam- 
ple, a mapping to an angle in radians was 
chosen in the initialization; see Listing Five. 

So far we have defined what the real- 
time tasks do. Now we need to set them 
up such that they run in real time within 
reasonable time constraints. Since this 
robot example is time triggered, the tasks 
are to run at regular intervals and poll the 
hardware. First, a PeriodicTimer event is 
set up that fires every 5 milliseconds. Then 
a periodic Task is created based on that. 
The Task has a duration of 2 millisec- 
onds— no allowance for an error handler 
and a deadline, which is the same as the 
period; see Listing Six. 

Because the Planner does trigono- 
metric calculations, it runs at twice the 
period of the Control task, namely 10 
milliseconds. To do this cleanly, Jbed of- 
fers a special type of task called “har- 
monic” task. The corresponding Real- 
timeEvent is a HarmonicEvent that 
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(continued from page 83) 

triggers at every second start of the Con- 
troller. With these six statements, you 
have set up the entire real-time system. 


The ADAPT Project 

The Advanced Diagnosis with the Applet 
Technology (ADAPT) project is a European 
Union-supported collaboration between Me- 
cel (a Swedish company that develops elec- 
tronics and software for the automotive in- 
dustry; http://www.mecel.se/) and Trialog 
(a French software house that focuses on 
embedded systems and home/vehicle con- 
nectivity solutions; http://www .trialog.com/) 
that aims to develop methods for remote 
downloading of software to cars. 

As cars become more and more com- 
plex, much of their functionality is con- 
trolled by electronic control units (ECUs). 
These ECUs are connected and commu- 
nicate with each other in networks. When 
an error occurs somewhere in the net- 
work, the symptom can occur at a total- 
ly different place. Thus, it is hard to local- 
ize faults in new cars and that is why a 
good diagnostics system has become more 
and more important. The main point is to 
be able to download and run Java-based di- 
agnosis components (Diaglets) via, for ex- 
ample, a diagnostic web site. 

To enable this, at least one of the ECUs 
in the car has to support code download- 
ing. In the ADAPT project, a telematics unit 
(TU), with GSM modem, is used inside the 
car as a communication link between the 
diagnostics web site and car. The TU uses 
Jbed as the operating system, which has a 
built-in Java virtual machine VM), and 
code can dynamically be loaded and un- 
loaded via GSM. It will also be possible to 
download diaglets from the TU to other 
ECUs inside the car if those ECUs have 
JVMs. In the ADAPT project, all parts of a 
complete system for remote car diagnos- 
tics are taken care of— programs for the 
diagnostics web site, all embedded pro- 
gramming inside the cars, security and safe- 
ty aspects, and so on. To simplify devel- 
opment, all programming is in Java. 


Device Drivers in Java 

One of the most fundamental qualities of 
the ADAPT system is dynamic down- 
loading of code, which is a feature that 
the Java language in itself supports. That 
was one of the main reasons why we 
chose Java in the first place. After we de- 








Table 1: Jbed real-time tasks. 
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cided to use Java, our next decision was 
whether to write the device drivers in C 
and use a native interface between C and 
Java, or write them in Java. C is often seen 
as the natural choice for drivers because 
it allows tight control of the hardware, but 
Java drivers have some advantages. The 
most obvious is that you have one con- 
sistent code base that can be manipulat- 
ed with one set of tools, instead of hav- 
ing two different sets of code with two 
different sets of tools. There is also no 
need to have any awkward half-Java/half- 
C interfaces between the two. 

Java has some other advantages com- 
pared to C. For example, javadoc is a built- 
in documentation system that produces doc- 
umentation in HTML format. Of course, tools 
like this exist for C, but it is a standard tool 
in Java and you can count on it being there. 

Also, Java produces cleaner code and less 
room for errors by having features such as 
strong typechecking, garbage collection, ex- 
ceptions, and no pointer arithmetic. 

Because of the safety checks Java makes 
during run time, Java is slower than C, even 
when compiled (of course, this depends on 
your compiler). Listing Seven shows some 
of the assembly code generated by the com- 
piler for the innermost loop in one of our 
communication-bus-driver interrupts. This 
code is performance critical since failure to 
execute in time leads to loss of data. In List- 
ing Seven, the extra lines of code added for 
safety checks are highlighted. These add 
20-25 percent to the execution time. But 
the safety checks increase stability and re- 
duce debug time, so they are a definite ad- 
vantage for all but the most speed-critical 
code. In Jbed, you can turn off run-time 
checks on a per function basis. The possi- 
bility to turn them off in performance- 
critical parts of the code gives us the best 
of both worlds. 

Another tricky area when writing device 
drivers involves interrupts. Jbed can han- 
dle interrupts in two different ways. The 
old-fashioned way has a pointer to the 
interrupt-handler function, which executes 
immediately and bypasses the normal 
scheduling of threads and tasks. Jbed dif- 
ferentiates between normal Java threads, 
which execute with no real-time require- 
ments, and real-time tasks that are guar- 
anteed to execute within a deadline.) In 
the other scenario, Jbed can schedule the 
interrupt as a real-time task that cooper- 
ates with the other threads and tasks. Tasks, 
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however, cannot be scheduled with greater 
granularity than the minimum timeslice — 
in our case, a 22-MHz 68376, which was 
set to 1 millisecond. Because the interrupts 
from the CAN bus could come as often as 
every 200 us (at 500 Kbaud), we were 
forced to use the traditional approach. 

Having interrupt handlers preempting 
the normal threads and tasks creates an- 
other set of problems: 


e You cannot expect the tasks to deliver 
on any real-time demands when they 
can be interrupted at any moment and 
for an arbitrary amount of time. 

All Java functions that trigger a 
rescheduling (such as wait and notify) 
cannot be called from the interrupt han- 
dler. Calling any such function would 
make the scheduler continue executing 
a thread or task and you would never 
be returned to the interrupt handler, which 
would leave your stack a permanent mess. 
You cannot allow any exceptions to be 
thrown in the interrupt handler, as this 
would also result in a messed-up stack. 
This problem is another reason you would 
turn off the safety checks. No safety 
checks means no stack-messing interrupts. 


In ADAPT, we turn off the safety checks 
only for the most performance-critical parts 
of our drivers and the interrupt handlers. 

When writing device drivers, you must 
have read/write access to memory. Nor- 
mally, Java does not support direct memo- 
ry access, but some kind of support for that 
is obviously needed. The Jbed operating 
system supplies you with some built-in 
functions for memory access. To increase 
the safety of those functions, we encapsu- 
lated them once more by creating a set of 
classes that represents bits, bit-fields, and 
fields at byte-sized and short-sized memo- 
ry locations. Listing Eight is one of those 
classes. It could, for example, be used to 
represent a value consisting of the bits 4 to 
6 (a mask value of 0x70) in a byte-sized 
hardware register. Using classes like this 
gives a reasonable level of safety to an oth- 
erwise completely unsafe write operation 
directly to memory. To achieve this and to 
keep the performance overhead low, as 
much checking as possible is done in the 
constructor, which is only called once. 
When the memory-write function executes, 
only a simple bounds check is performed. 
And because we can be sure that the vari- 
ables in the functions are correct, you can 
turn off the built-in safety checks for an 
even smaller performance hit. And with Java 
exceptions, we can, in many cases, grace- 
fully recover from situations that could oth- 
erwise lead to a system crash. 
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e 7 
Listing One | 
public class Task extends java.lang.Thread { 
public static final int timeslice; 
public Task (java.lang.Runnable target, long duration,long allowance, 
long deadline, RealtimeEvent event) throws AdmissionFailure; 


} 
e e 
Listing Two 
// some real-time constructor is called: 
com. jbed.tasks.RealtimeEvent event = new com.jbed.tasks....(...); 


// some object implementing the java.lang.Runnable interface is created: 
java.lang.Runnable target =... ; 
com. jbed.tasks.Task task; 
try { 
task = new com.jbed.tasks.Task(target, duration, 
allowance, deadline, event) ; 


} 

catch (com. jbed.tasks.AdmissionFailure e) { 
} 

task.start; 


Listing Three 


public class LPointer { 


// the two clocks in ms: 
static final int ControllerClock = 5; 
static final int PlannerClock = 1@; 


// the calculation parameters: 
static final float Kp®@ = 500.@F, Kpl = 600.@F; 
static final float Kd®@ = 35.@F, Kd1 = 10.QF; 


// the Lissajou parameters: 

static float a® = Q.05F, al = O.O5F; 
static float £@ = 12.56F, f1 = 6.28F; 
static float fv® = @.OF, fvl = O.OF; 
// actual positions and previous positions: 
static float q@actual, qiactual, q@old , qiold; 


// the three tasks: 
static com.jbed.tasks.Task controllerTask, plannerTask, watchdogTask; 


// the periphery (actuators and counters): 
static com. jbed.peripherals.AnalogActuator out@, out1; 
static com.jbed.peripherals.Counter counter@, counter1; 


// the desired positions: 
static float q@desired, qidesired; 


// the time: 

static float t=0; 

Oi ees 

static void init() { 
oe 
try { 


// Configuration 

float gear = -10@F; 

float nmPerAmp = @.@5F; 

float maxAmp = 6.QF; 

float maxNm = maxAmp * nmPerAmp * gear; 
float scale = 2 * maxNm / 4096; 


// define the scale for the actuators: 
com. jbed.drivers.LinearScale.defLinearScale("DAScale", 

®, 4095, scale, maxNm) ; 
drivers.demo.DAC.defModule("DACMod", @x@FFFF8190) ; 
drivers.demo.DAC.defChannel("AQ", "DACMod", "DAScale", 3); 
drivers.demo.DAC.defChannel("Ai", "DACMod", "DAScale", 4); 


// define the scale for the input counters: 

float counterConst = 2000; // = 4 * 500 

scale = 2 * com.jbed.math.FMath.PI / (counterConst * gear); 
com. jbed.drivers.LinearScale.defLinearScale( 

"EncScale", Integer.MIN_VALUE, Integer.MAX_ VALUE, scale, @F); 
drivers.demo.Enc6.defModule("Enc6Mod", O@x@FFFF83@Q@) ; 
drivers.demo.Enc6.defChannel("Counter@", "Enc6Mod", "EncScale", 3); 
drivers.demo.Enc6.defChannel("Counteri", "Enc6Mod", "EncScale", 4); 


// Initialization 

out® = new com. jbed.peripherals.AnalogActuator ("AQ") ; 
outi = new com. jbed.peripherals.AnalogActuator("A1") ; 
counter® = new com. jbed.peripherals.Counter("Counter@") ; 
counterl = new com. jbed.peripherals.Counter("Counter1") ; 


} catch (Exception e) { 


} 
LP wes 
} 
Ieee 
} 


e e 
Listing Four 
static class Planner extends com.jbed.tasks.Task { 
public Planner(long duration, long allowance, long deadline, 
jbed.tasks.RealtimeEvent event) 
throws com.jbed.tasks.AdmissionFailure { 
super (duration, allowance, deadline, event); 


public final void run() { 
q@desired = (a@*com. jbed.math.FMath.sin(f@*t+fv@) ) ; 
qidesired = (ai*com. jbed.math.FMath.sin(f1*t+fv1)); 
t = t+(float) (PlannerClock/1000.@F) ; 
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Listing Five 


static class Control extends com. jbed.tasks.Task { 
public Control(long duration, long allowance, long deadline, 
jbed.tasks.RealtimeEvent event) 
throws com.jbed.tasks.AdmissionFailure { 
super (duration, allowance, deadline, event); 


public final void run() { 
float taud, taul; 
float dpos@, dposi; 


q@actual = counter@.read(); 
qiactual = counteri.read(); 
dpos® = q@desired - qOactual; 
dposi = qidesired - qlactual; 


tauS = KpO@*dposB-KdO* 
((q@actual-q@old) / (float) (ControllerClock/190@.@F) ) ; 

taul = Kp1*dposi-Kd1* 
((qlactual-qiold) / (float) (ControllerClock/1000.@F) ) ; 

out@.write(tauS); outi.write(tau1) ; 

qGold = q@actual; qiold = qlactual; 


e e e 
Listing Six 
com. jbed.tasks.RealtimeEvent periodicEvent = 
new com. jbed.tasks.PeriodicTimer (ControllerClock*1@@Q) ; 
controllerTask = new Control(20@0, 9, ControllerClock*10@0, periodicEvent) ; 
controllerTask.start(); 


com. jbed.tasks.RealtimeEvent harmonicEvent = 

new com. jbed.tasks.HarmonicEvent(controllerTask, 2); 
plannerTask = new Planner(1@08, 0, 10000, harmonicEvent) ; 
plannerTask.start(); 


° ° 
Listing Seven 
OO1FOF66 
MOVEA.L (-16, A6), A@ 
QG1FOF6A TST.L Ad 
@O1FOF6C TRAPEQ 


@Q1FOF6E MOVEA.L (12, A®), AO 
Q@1FOF72 MOVE.L (-8, A6), D1 
Q@1FOF76 ADD.L (-20, A6), D1 


Q@@1FOF7A MOVEA.L Di, Al 
Q@1FOF7C MOVE.L (-20, A6), DO 
QO1FOF8O TST.L Ad 

OO1FOF82 TRAPEQ 

QOO1FOF84 CMP.L (12, A®), DO 

@O1FOF88 TRAPCC 

Q@1FOF8A MOVE.B (-3834, Al), (16, AO, DO.L¥1) 
Q@@1FOFI@ ADDQ.L #1, (-20, A6) 

@O1FOF94 CMPI.L #$00000008, (-20, A6) 
Q@1FOFIC BLT SOO1FOF66 


Listing Eight 


public class ByteRegisterField 
{ 


private int m_mask; 
private int m_base; 
private int m_pos=0; 


private int m_maxval; 
private static IllegalArgumentException IllegalArgument = 
new IllegalArgumentException() ; 
public ByteRegisterField(int base_adress, int mask) 
{ 


m_mask=mask; 
m_base=base_adress; 
//check to see that the mask is in the register and nonzero 
if (m_mask>OxFF | |m_mask<=@) 
throw IllegalArgument; 
//Find the lowest bit in the mask 
int i=1; 
while ((m_mask&i) ==0) 
{ 
i<<=1; 
m_postt; 


m_maxval=m_mask>>>m_pos; 
//check to see if the mask is without holes; 
if (((m_maxval+1) &m_maxval) !=0) 

throw IllegalArgument; 


public void Set(int value) 


//instruct the compiler to turn of safety checks 
com. jbed.runtime.Unsafe.option(com. jbed. runtime.Unsafe.noChecks) ; 
//check that the value is valid 
if ((value > m_maxval) |jvalue <@) 
throw IllegalArgument ; 
Unsafe. getByte(m_base) & ~m_mask | (value << m_pos))); 


public int Get() 
//instruct the compiler to turn of safety checks 


com. jbed.runtime.Unsafe.option(com. jbed.runtime.Unsafe.noChecks ); 
return (com. jbed.runtime.Unsafe.getByte(m_base) & m_mask)>>>m_pos; 
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s the Internet becomes ubiquitous, 
software is becoming increasingly 
exposed to complex and dynamic 
environments. Consequently, the 
concept of agent-based computing has 
emerged as a useful model for software 
that has to deal with an uncertain and 
changing environment that consists of 
many distributed software systems. 

Agents are software entities that not only 
gather and present information, but also 
act intelligently to accomplish one or more 
assigned goals. Agents can often be de- 
scribed as finite state machines whose 
state transition dynamics drives them to 
achieve the goal. 

A state-machine model whose state tran- 
sition dynamics has to be entirely hardwired 
and manually maintained offers little value 
to programmers. In this article, We'll exam- 
ine the application of machine-learning tech- 
nology to control an agent. Based on Markov 
process, this technology enables an agent to 
automatically identify the statistical structure 
of its environment and synthesize optimal 
feedback strategies at run time. As the re- 
sult, a software agent powered by this tech- 
nology will be able to adapt to its environ- 
ment and continuously learn and improve 
its ability to achieve the goal. To this end, 
we present MLEngine, a general-purpose 
AI engine with real-time learning capabili- 


Zhimin is the founder of the Learning Ma- 
chines Technology Group and can be con- 
tacted at learning@lmtg.com. Li is a soft- 
ware design engineer at Hewlett-Packard. 
She can be reached at liliu@hpl.bp.com. 
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ty. MLEngine can be used to control simu- 
lated game characters or software agents 
through interface with (simulated) sensors 
and actuators. MLEngine can synthesize goal- 
oriented behavior at run time and therefore 
make the characters under its control learn. 
MLEngine, available from both DDJ (see “Re- 
source Center,” page 5) and at http://www 
.Jmtg.com/, includes Win32 run-time li- 
braries, API documentation, sample source 
code, and demo programs. 
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Agent Dynamics 

Again, the dynamics of an agent can of- 
ten be conveniently described as a state 
machine. The basic elements of a state 
machine are: 


e At any instance, the dynamic state of an 
agent can be uniquely described by a 
set of variables called “state variables.” 
These variables can be analog or dis- 
crete. 

e An agent can receive input stimuli from 
its environment, and the state variables 
will undergo changes. The new state is 
a function of both the environment in- 
put and the previous state. 

e An agent can pass stimuli back to its en- 
vironment. The output is uniquely de- 
termined by the state variables. The en- 
vironment could include other agents. 
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If the states are discrete and finite, this 
mechanism is called a “finite state machine.” 

An intuitive way to illustrate the idea is 
to use a game character such as that in Fig- 
ure 1. Suppose that as this game character 
moves around in space, you could use its 
position in space to describe its dynamic 
state. Additionally, a character usually has 
an orientation. An orientation indicator can 
serve as another state variable. Imagine 
that this character can also sense whether 
it is in touch with an obstacle. Touch-sens- 
ing input can also become a state vari- 
able. So to completely describe the char- 
acter, you need to specify its position, 
orientation, and whether or not it is in 
touch with an object. 

The state variables undergo changes 
over time. For example, the character can 
move in space and flip. The way it moves 
can be expressed as transitional proba- 
bility between states. Obviously, the agent 
state transition will be influenced by in- 
puts from its environment under certain 
conditions, so that it will not become a 
completely self-indulging entity. For ex- 
ample, the character’s position change 
depends on its environment, such as 
whether or not an obstacle is present. 

The character's state should also be able 
to undergo changes without any environ- 
ment stimuli under certain conditions (so 
that the new state depends only on its pre- 
vious state). This is important for imple- 
menting any oscillatory or self-driven se- 
quencing behavior such as walking in a 
legged character. 

Assume that you are interested in an 
agent whose behavior could evolve as it 
adapts to its environment. A sensible ap- 
proach toward that goal is to separate con- 
straints from voluntary control in the agent 
design. A suitable analogy is the human 
model. Human behavior is determined 
partly by the brain and partly by the con- 
straints imposed by the physical body (for 
example, you couldn’t jump 100 feet in 
the air even if you wanted to). 

In Figure 2, you divide an agent into a 
physics block and a control block. The 
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(continued from page 88) 

physics part of the agent is a state machine. 
The controller has access to state informa- 
tion and issues a set of actions. Along with 
the other inputs, the action input will in- 
fluence state transitions. So now the state 
transition is determined not only by exter- 
nal input and the previous state, but also 
the action input from the controller. 

The character in Figure 1, for example, 
has five movement actions— left, right, 
up, down, and flip. The probability of the 
character falling into any state depends 
not only on its previous state, but also on 
the environment input as well as the vol- 
untary action input from the controller. 


Setting Up the Goal 
Before learning can take place, it is nec- 
essary to come up with a formal definition 
of a goal. After all, the purpose of learn- 
ing is for the agent to achieve the goal. 
A convenient way to define a goal is to 
choose one or more states and assign 
them as the goal. The fact that the goal is 


expressed as one or more states means_ 
that the goal has to be a part of the state = 


space that can be sensed by the contioller. 
Intuitively, this makes sense because no 
matter how intelligent an agent is, it would 
never be able to learn and improve if it 
cannot sense whether or not it has 
achieved the goal. 

To make learning happen, you have a 
few obvious and tempting choices: 


e You simply observe and see which ac- 
tions cause the state transitions that lead 
to the goal. You then enhance the prob- 
ability of taking these actions. This is es- 
sentially a conditioning based on reward 
and penalty. You reward the actions that 
bring favorable results and penalize 
those that do not. 

The problem with this approach is 
that learned behaviors tend to be too 
simplistic. The character can only learn 





Figure 1: An illustrative example of 
an agent. 


Dr. Dobb’s Journal, November 1999 


simple reflexes that seek immediate re- 
wards. It will not learn to plan ahead 
and carry out sequential behaviors. 

e You try different combinations of fea- 
tures in the feedback strategy and see 
which one yields the best overall result. 
If we try long enough, a good combi- 
nation of features in the controller can 
be found. And the character could be- 
come very smart. 

Experience suggests that combinato- 
rial approaches only work well for sim- 
ple problems. As the state space grows 
big, the amount of time needed to per- 
form trial-and-error quickly becomes in- 
tractable, even if it is done in a simula- 
tion environment. 


Mathematical Analysis of Learning 

For convenient analysis, you can encode 
the states in the state space into a linear 
array 1,2,3...1, assuming there are a total 
of m mutually exclusive states. 

Once the states are encoded into a lin- 
ear array, the state transition can be de- 
scribed by an 2Xn matrix. Since the state 
transitional probability depends on the ac- 
tion input from the controller, to com- 
pletely describe the physics of a charac- 
ter you need m such nxn matrices, 
assuming there are m actions. In Exam- 
ple 1, for instance, 

k 
a 
represents the transitional probabilities 
from state 7 and state j/ under action k. If 
the state transition is fully deterministic, 
then a;; is either 1 or 0; otherwise, it is 
some positive value in between. 

In Example 2, the controller can be de- 
scribed by an nxm matrix, where x;; is 
the probability of taking action 7 when the 
character is in state 7. 

The combined system that includes both 
the physics and the controller can be de- 
scribed by a transitional probability ma- 
trix 7 (see Example 3), where 


y (Rk) 
ty= 2 xy Gy 
Suppose that on average, the state proba- 
bility can be described by P={p1,)2,...Dy}, 
where p; is the longterm average probabil- 

ity for the system to be in state 1. 
According to Markov process theory, 
under a certain condition (the ergotic con- 
dition), the state probability will converge 
to a stable value that satisfies P=PeT. 
Suppose that the goal is state 0, the pur- 
pose of learning is to enhance P, by ma- 
nipulating control variables. Obviously, you 
cannot directly manipulate P, which is the 
consequence rather than the cause of the 
system behavior. You can, however, ma- 
nipulate transition probabilities t;; through 
Xjp. This will, in turn, influence’ P in the 
long term, based on the relationship P=PeT. 
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(continued from page 92) 

The advantage of a run-time approach 
is that the controller is adaptive. If the 
physics of the character and its environment 
undergo changes, the controller will reor- 
ganize and adapt the changes. The end re- 
sult is that the character can maintain and 
improve its capability to achieve its goals 
in response to environment changes. 

Here is how system identification works. 
The controller issues actions and observes 
changes in state. If action k is issued and 
the state undergoes a change from / to /, 
you will make adjustments to 

(k) 

aj, 
in Example 4, where A is the learning rate. 
It is usually a small positive number less 
than 1. For all the other 


n-1q® 
tJ 


with 7'47, you make adjustments accord- 
ing to the amount in Example 5. If you 
keep doing that, over a period of time, 
matrices 

k 

a” 

j 
converge to best represent the state tran- 
sition probabilities. 


Putting It Together 
Based on this analysis, you have put to- 
gether an agent controller such as MLE, 
which can be integrated into an agent and 
make it capable of learning. The learning 
is carried out in stages. 

In the explore phase, MLE drives the 
agent to interact freely with its environ- 
ment. MLE passes out quasirandom ac- 
tion commands and monitors the state 
input. As a result, the agent will start to 
explore its environment. Internally, MLE 
associates sensory and action informa- 
tion and builds an internal representa- 
tion of the agent and its environment. 









Example 3: The combined system. 






Example 4: Making adjustments. 





Example 5: Making further 
adjustments. 
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This is essentially a nonmodel-based sys- 
tem identification process. 

A parameter called “association entropy” 
can be obtained from MLE by calling 
GetAssociationEntropy. This value (which 
ranges from 0 to 1) should go down as 
the association phase is being carried out. 
The smaller this value is, the more deter- 
ministic MLE has become while it builds 
an internal representation of the system 
under its control. It represents the confi- 
dence level of the agent in terms of its un- 
derstanding of the environment. 

In the optimization phase, MLE will syn- 
thesize a strategy, which drives the agent 
to achieve the goal defined by SetGoal. 
The strategy is essentially a sensory feed- 
back controller that determines how the 
system should react to sensory inputs and 
interact with its environment. 

A parameter called “optimization en- 
tropy” can be returned from the MLE by 
calling GetOptimizationEntropy. This val- 
ue should go down as the optimization 
phase is being carried out. The smaller 
this value is, the more deterministic the 
synthesized strategy is. The estimated 
probability to achieve the goal can be ob- 
tained from GetGoalProbability. 

The three phases can be switched back 
and forth by monitoring association en- 
tropy, optimization entropy, and the goal 
probability. For example, during the op- 
timization phase, the application can keep 
statistics of the average probability of 
achieving the goal and compare it with 
the estimated goal probability returned by 
GetGoalProbability. If the two are suffi- 
ciently different, the application can start 
to explore and associate again, during 
which the association entropy is moni- 
tored. Once the association entropy goes 
down to a stable value, an optimization 
phase can be started. The optimization 
entropy should go down while the goal 
probability will go up; see Figure 3. 

We've included two demos (available 
electronically) of agent learning. The first 


Associate 





is a boxer trying to hit its opponent. At 
first, the opponent does not really respond 
well to the boxer’s moves. The boxer 
learns to carry out coordinated arm move- 
ment to hit the opponent repeatedly at the 
highest frequency. Later, we change the 
opponent's strategy by making it more re- 
sponsive. The opponent moves his fist to 
block the boxer’s punch. After some ad- 
justment, the boxer learns to fake fist 
movements and trick the opponent to 
move its fist to the wrong location, and 
then hits from another direction. 

The second demo is Picker, a cartoon 
character whose goal is to eat the fruit at 
various locations. It learns to associate its 
visual information with its movement and 
approach the fruit efficiently. You could 
cause Picker to mutate so that its mouth 
and its eye will be located on a different 
side of the body. The picker learns to flip 
around so that it can see where the fruit 
is and approach it with its eye facing the 
fruit. Once there, it turns around again 
and eats it. For a Pentium-class PC, it usu- 
ally takes one or so minutes to learn a 
new strategy if the size of the state space 
is less than 50. 

We have also tried MLE with (analog) 
simulations that faithfully emulate physics. 
For example, we coupled MLE with a 
quadruped simulator that simulates hu- 
man body mechanics. we set the goal as 
the model making forward motion while 
maintaining an upright body position. The 
model learned to walk and run realisti- 
cally like a human (see Figure 4). 


The MLE API 

Based on the aforementioned method, 
We've designed a general-purpose AI en- 
gine called MLEngine (MLE); its interface 
functions are presented in Listing One. To 
prepare the character for learning, you 
need to follow these steps: 


1. Design the character’s physics. 
2. Encode the state information into a 





Optimize 


Goal 
Changed? 


— Environment 
mChanged?qe 





Figure 3: MLE flow chart. 
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Figure 4: Agent controller coupled 
with real physics to model human 
locomotion. 


(continued from page 94) 
linear array ranging from 0 to dim_state 
-7. For example, if an agent has three 
state variables, each represents three 
possibilities. It is sufficient to represent 
the dynamics of the agent into 27 dis- 
crete, mutually exclusive states (or less 
since some combinations may not be 
possible). 

3. Create an instance of MLEngine. Spec- 
ify the dimension of the state space and 
the action space. 

4. Select one or more states as the goal 
and call SetGoal to set them up. In the 
boxer example, the goal is the state 
when the arm is extended and the fist 
is landed on the opponent's body. 

. Connect the controller to the character 
physics by calling Perform, which ac- 
cepts state information as the input and 
pass Out actions. 


WW 


The learning process is carried out in 
three phases (see Figure 3). Phase con- 
trol is accomplished by calling SetPhase. 


Comparison with Other Approaches 
At the beginning of AI, intelligence was 
thought of as the ability to manipulate 
symbols and abstract concepts. An intel- 
ligent system based on this assumption 
consists of a reasoning engine and 
database that stores common sense knowl- 
edge (an expert system, for example). To 
make it useful, real-world information has 
to be mapped into abstract concepts or 
symbols before machine reasoning can 
take place and the expert system database 
can be updated. The task of recognizing 
real-world patterns and mapping them 
into concepts often turns out to be pro- 
hibitively difficult. 

Symbol-based automatic reasoning is 
becoming less popular these days because 


it never produced interesting results. Re- 
cently, a school of thought has emerged 
that believes logical reasoning is more an 
afterthought that justifies actions that are 
already taken, than the cause of actions. 
For example, when you face a situation 
and make a decision, it is often after you 
have made that decision that you come 
up with the thought: “I did that because 
of such and such reasons...” We all have 
a tendency to consciously justify what has 
been decided unconsciously. 

The literature on neural networks is 
vast. The most popular models focus on 
pattern association and classification. For 
example, back-propagation models can 
map a set of input patterns into some 
output patterns according to a set of ex- 
amples. A back-propagation neural net 
can be used as a controller that maps 
sensory input patterns into actuator out- 
puts provided one supplies examples 
about what to do under all situations. 
The controller can only generalize from 
these examples. 

A second class of neural net models can 
learn to categorize analog patterns into 
discrete clusters without supervision. The 
purpose of learning is to reduce encod- 
ing error. The best known example is the 
Kohonen neural net model. 

There is yet another class of neural net 
models that are able to generate sequences 
of spontaneous patterns. The output of 
these neural net models can be used to 
control motor actions. Most of these neu- 
ral net models have recurrent topology and 
dynamics (Hopfield neural networks and 
Bolzman machine, for instance). Markov 
process theory has often been used to 
mathematically describe the dynamics of 
recurrent structures. Our learning algorithm 
is similar to some aspects of these mod- 
els, but only at the mathematical level. 

Learning automata are a set of mathe- 
matical models that address learning di- 
rectly without worrying about the under- 
lying brain substrate. The heart of this 
approach is conditioning based on reward- 
and-penalty. The main problem with this 
class of methods is that the results are usu- 
ally too simplistic. For example, a typical 
learning automaton can only learn to per- 
form actions that will directly lead to re- 
ward. It will not learn to plan ahead and 
accomplish the goal through indirect ac- 
tion sequences. 


More sophisticated learning automata 
are being developed today. Most of them 
try to give credit to actions that indirect- 
ly contribute to achieving the goal. Some 
of these models borrow ideas from neu- 
ral network models. 

It is said that if you give a monkey a 
typewriter and allow it to type randomly, 
you will eventually see beautiful poems 
coming out of that typewriter. Many tough 
problems can indeed be solved by ran- 
dom searches in the solution space if we 
do not worry about the time it takes. Sim- 
ulated annealing and genetic algorithms 
(GA) are two such techniques. Combina- 
torial optimization techniques such as GA 
and simulated annealing search through 
the control parameter space to optimize 
the performance measured by a prede- 
fined index or criterion. The search is 
usually guided by some sort of heuris- 
tic. For example, simulated annealing 
uses gradient information to guide the 
search. GA recombine simulated gene 
segments to come up with control pa- 
rameters for new trials. 

The main problem with this class of ap- 
proaches is the computational requirement. 
For example, if you use GA to optimize 
the performance of a game character, the 
algorithm has to simulate the game char- 
acter through many generations. In each 
generation, a large population of individ- 
ual characters has to be simulated also. The 
algorithm will come up with many sets of 
control parameters for those individual char- 
acters. Each of these characters has to per- 
form long enough to come up with a fit- 
ness measurement. A character will not 
adapt in its lifetime. Combinatorial algo- 
rithms are fundamentally difficult to im- 
plement in real time. 


Conclusion 

There is not a one-size-fits-all solution for 
machine learning. A practical solution has 
to inherit the best of what is available. Learn- 
ing as an optimization algorithm may take 
many forms. A successful one converges to 
an optimal result not because it is a neural 
net, learning automata, or whatever else, 
but because its underlying mathematics is 
correct. It is therefore best to deal with learn- 
ing at the mathematics level. 
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Listing One 


typedef state int; 
typefef action int; 


void SetPhase(int) ; 


void Associate(); 


double GetAssociateEntropy(); 


void Optimize (); 


class MLEngine { 


: double GetOptimizeEntropy() ; 
public: : ; ; ; ; ; ; double GetGoalProbability() ; 
MLEngine(int dim_sate, int dim_action, int dim_order) ; 
“MLEngine() ; void SetAssociationRate(double r); //O<r<1 


void SetOptimizationRate(double r); //O<r<1 
void SetGoal (state) ; //one of the states is assigned as the goal } 
void SetGoal(state*, int); //when multiple states are assigned as the goal 


action Perform(state) ; //@--explore, 1--associate, 2--optimize and perform 
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Trends in High-Tech 
Hiring Practices 





Finding your way 
in a rough and 
tumble job market 





Reginald B. Charney 


iring practices in the information 
technology CIT) industry are chang- 
ing more rapidly than ever before. 
To make sense of this rapidly 
changing employment landscape, I com- 
piled— then analyzed— a database of 
over 100,000 new technical jobs listed 
on the Internet. In 90 days, from the be- 
ginning of May 1999 to the end of July 
1999, an average of 219 new technical 
positions were listed daily. Of this num- 
ber, 114 were for positions as software 
engineers, which accounts for more than 
52 percent of all new IT jobs listed. In 
that period, the number of technical po- 
sitions advertised rose 17.4 percent, while 
software engineering openings rose 49 
percent. 

In this article, I present a number of 
charts that illustrate some trends my anal- 
ysis reveals. While the charts focus on 
technical skills that are in demand, op- 
portunities also exist in areas such as man- 
agement and marketing. Consequently, I'll 
touch on those areas, as well as share 
some pointers on maneuvering in today’s 
job market. 





keg is president of stats2g0.com. He is also 
a voting member of ANSI’s X3]16 Com- 
mittee on C++. He can be reached at 
reg@stats2go.com. 
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Technical Skills Are in Demand 

The charts I present show the demand 
growth rates for skills— not the total num- 
ber of positions available. For example, if 
there was a demand for six Eiffel software 
engineers at the start of May and a de- 
mand for 12 at the end of July, then de- 
mand for Eiffel skills increased 100 per- 
cent. On the other hand, if during the 








































































































same period the demand for Java software 
engineers increased from 5000 to 6000, 
then the growth rate is only 20 percent. 
Negative growth rates indicate shrinkage 
in demand for that skill. 

In developing these charts, I've made a 
number of assumptions. First, the trend lines 
have been treated as linear. Also, this sam- 
ple is representative of the job market in 
general. Lastly, we know that hot topics like 
Linux and Java have nonlinear growth rates, 
but for this short a time period, a linear as- 
sumption provides a good estimate. 

Figure 1(a), for instance, shows that jobs 
for PL/1, Cobol, and SmallTalk program- 
mers are losing ground as demand for 
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them dwindles. On the other hand, de- 
mand is growing fastest for XML/HTML, 
Eiffel, Perl, and Java developers. 

Figure 1(b) illustrates the normalized 
demand for each language. Here, the mar- 
ket growth/shrinkage is multiplied by the 
proportion of the market the language has. 
At the moment, more opportunities 
abound in Java and C/C++ than in any 
other language. 

Figure 2(a) shows that the growth rate 
for Linux skills is outstripping demand for 
any other platform. Meanwhile, demand 
is up for skills on all major platforms. 
(Both UNIX and Macintosh trends are 
based on regression calculations, not on 
actual figures.) 

In Figure 2(b), it appears as if the cur- 
rent demand for Windows NT, and Win- 
dows in general, far outweighs any other 
platform. However, Solaris and Linux both 
make a respectable showing. Especially 
since the latter has only recently received 
a lot of media attention. (Both UNIX and 
Macintosh normalized demands are based 
on regression calculations, not on actual 
figures.) 

Figure 3(a) illustrates the changing de- 
mand in different technologies. The 
growth rate is greatest for ASIC-, database, 
CORBA-, and Internet-related skill sets. 
Internet-related skills are defined as po- 
sitions listing Internet, Java, HTML, DHML, 
or XML in their job listings. 

Finally, Figure 3(b) shows that the great- 
est demand is in Internet-related tech- 
nologies. Database skills come in a re- 
spectable second. While the first is no 
surprise, the fact that database skills are 
in high demand may be. 


On the Management Front 


It is no secret that, in all business areas, 
outsourcing is becoming a way of life. 
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There are plenty of graduates 
with good grades, but we’re 
looking for students with 
dreams and ambition. People 
who want to take on challenges; 
build jetliners, fighter planes, 
the Space Station. People who 
want to be an integral part of 
the projects they work on. If 
you re studying engineering, 
computing, or another technical 
discipline and have big dreams 
to go along with your degree, 
submit your resume to Boeing at 
www.boeing.com/employment 
or mail it to The Boeing 
Company, P.O. Box 3707, 
M/C 6H-PR, 003705, 
Seattle, WA 98124-2207. 


It is the policy of The Boeing Company 
to attract and retain the best qualified 
people available without regard to race, 
color, religion, national origin, gender, 
sexual orientation, age, disability, or status 
as a disabled or Vietnam-era veteran 


www.boeing.com/employment 











(continued from page S2) 

Companies are doing this so that they can 
concentrate on their “core competencies” 
and become more efficient. All other func- 
tions are being farmed out to contracting 
organizations. This includes human re- 
sources, payroll, system maintenance, 
operations, and most low-tech jobs. La- 
bor and tax laws also make contracting 
as an individual activity more and more 
difficult. Firms do not want the poten- 
tial liability of hiring independent con- 
tractors, so they buffer themselves by us- 
ing outsourcing agencies as much as 
possible. In many high-tech firms, con- 
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tract staff compose 14 to 40 percent of 
the work force. Because many compa- 
nies are hesitant to reveal exact out- 
sourcing figures, these estimates are 
probably low. Also, the outsourcing trend 
will grow. In the future, expect it to ex- 
ceed 70 percent of the operating staff at 
many companies. Only the timeframe is 
in doubt. 

Outsourcing also tends to turn contract 
staff into commodities. Because compa- 
nies are using outsourcing for noncritical 
tasks, the outsourcing agencies are look- 
ing for skilled workers who give reliable 
and consistent results. To avoid employ- 


100.0 150.0 


80.0 100.0 120.0 





Figure 2(b): Normalized demand by platforms. 
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ment liability, companies also may only 
keep contract staff for a short period and 
expect the outsourcing firm to supply re- 
placements on a regular basis. This em- 
ployment by the numbers also destroys 
any concept of company loyalty and the 
idea of commitment to a goal. More and 
more contractors will treat the job as a 9- 
to-5 experience. 

Outsourcing is not necessarily bad for in- 
dividual contractors. People working for 
outsourcing companies get more varied ex- 
perience and higher basic compensation 
than their directly employed counterparts. 
Contractors also get their marketing done 
for them by outsourcing firms, and con- 
tractors do not need to spend significant 
amounts of time finding the next contract 
nor dealing with the complex sales aspect 
of selling their services. 


In Marketing 

Marketing makes the world aware of your 
products and the potential solutions that 
your products and services offer the cus- 
tomer. To do that, marketing tries to find 
out what customers want. There are many 
ways of doing this, but the growth of the 
Internet has added new dimensions to this 
effort. It is now possible to get tens of 
thousands of potential customers inter- 
ested, where before marketing was limit- 
ed to dealing with hundreds at most. It 
used to take weeks, even months, to find 
out what was happening. It is now pos- 
sible to get useful information in hours or 
in a day or two. Given that marketing has 
found out what is needed, they then want 
to sell it on the Web using e-commerce. 
This will affect the basic technology we 
use and how we use it. Massive system 
rewrites will occur— not in the sense of 
modifying code, but in creating new sys- 
tems to replace old ones. The new sys- 
tems will also be expected to have greater 
functionality, interoperability, flexibility, 
efficiency, and reliability. 

Marketing may be the first to see the 
business opportunity for these new sys- 
tems, but expect the company to increase 
the pressure on its IT staff so that the com- 
pany can stay competitive. To deal with 
the explosive growth that the Internet 
brings, best-of-breed software will be pur- 
chased and deployed. So, besides need- 
ing to garner and improve your overall 
programming skills, expect to be trained 
in new externally developed products that 
your company will purchase/lease. 


Hiring Efficiency 

Job seekers are using the Internet more 
and middlemen less. At the same time, 
those hiring are becoming inundated with 
information and résumés. Both parties are 
also going nationwide using the Internet. 
Decisions on interviewing and hiring are 
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Figure 3(b): Normalized demand by technologies. 


speeding up —in more and more cases, 
you will hear in 1-2 days if you are in 
the pool of eligible candidates or not. 
The first interview is also likely to be a 
telephone interview. These trends are 
happening in many high-tech industries, 
not just computing. As Bruce Hatz, 
Hewlett-Packard Corporate Staffing Man- 
ager said, “Rapid response to qualified 
applicants is a competitive advantage.” 
Thus, the question becomes “Are you a 
qualified candidate?” 

The ideal candidate is a self-motivated, 
self-starting, team player who has just 
done exactly the desired job successful- 
ly in a closely related business— prefer- 
ably at the competition Gif those hiring 
you can avoid a lawsuit). Such a candi- 
date is ideal because they are the ulti- 
mate in efficiency: no training, immedi- 
ate utilization, highly recommended (by 
the competition), and no wasted effort 
on the part of the hiring manager trying 
to balance various tradeoffs. They are 
worth even more than they were getting 
before because they shorten and simpli- 
fy the hiring cycle, degrade the compe- 
tition’s competitive advantage, and add 
to the hiring manager’s reputation for at- 
tracting new people. 

Unfortunately, there never was nor 
ever will be an ideal candidate as de- 
scribed earlier. Any new employee is ini- 
tially inefficient: The new company’s cul- 
ture differs from the old one; the new 
team takes time to integrate any new 
team member; and the new firm’s pro- 
cesses and procedures differ from the 
old firm’s way of doing things. It will 
take any new employee three to six 
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months to integrate and become a part 
of any effective working group. Also, 
consider this: If the new person has done 
the same job at his previous employer, 
then he may not bring any new ap- 
proaches. Remember that this person was 
hired because he didn’t rock the boat— 
he was supposed to be plug-n-play. 

We are constantly faced with the trade- 
off between short and long term goals, 
between geeks up on the latest and great- 
est technology and those with a broader 
vision gained through experience. There 
is a place for both, but there are more 
slots for geeks than there are for the per- 
son with the vision that experience 
brings. You can deduce this from the fact 
that there is only one CEO at a compa- 
ny, but many individual contributors. So 
where does someone with broad expe- 
rience go? My recommendation is into 
middle management, project manage- 
ment, and mentoring. Or as one CEO of 
a rapidly expanding startup said, the old- 
er managers provide adult supervision 
for the younger employees. 

Middle management is currently a dirty 
word, but newer high-tech companies 
now face a dilemma. They have few 
management levels and the manager at 
each level tries to manage more and 
more people. Industrial psychology and 
organizational dynamics studies show 
that the ideal small group size is 7 +/- 2 
people (Miller's Law). Graicunas’ Law 
states that relationships increase expo- 
nentially with the number of people 
managed. Thus, a department larger than 
150-200 people becomes unmanageable. 
A department of 200 needs to be broken 
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up into 20-30 teams. Each team uses a 
team or project leader. A possible man- 
agement hierarchy could be one depart- 
ment head, 4—5 assistant department 
heads, 25-30 project leaders, and 164-170 
individual contributors. 


Prepare Yourself 
Here are a number of things you can do 
to actively manage your career: 


e Learn at least one new skill per year 
and use it so that the experience is real. 

e Network inside and outside your nor- 
mal environment. Volunteer for some- 
thing that interests you. 

¢ Be opportunistic. Learn about what is 
hot and what is not. 

e Find a mentor, be a mentor. 

e If you are transitioning up the organi- 
zation, keep your skills set current for 
the type of team you are managing— 
don’t give up your day job. 

e Keep your résumé as current as possi- 
ble. List skill sets, responsibilities, and 
projects as you work on them. 

e Be prepared to contract with an out- 
sourcing agency— this means keeping 
your skills current. 


Here are some things to avoid: 


e Working all hours. Better you should 
“get a life.” 

¢ Becoming complacent about your skills, 
position, or network. 

¢ Believing any job is permanent. 

e Being so self centered that you find 
yourself often working alone. 


Conclusion 

You know that change is coming. Be pre- 
pared. Train yourself, remain flexible, and 
network like mad. Treat the changes that 
do come as opportunities to see new vis- 
tas and, lastly, enjoy the journey. 
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ecause of the shortage of trained soft- 
ware developers, many companies 
rush to recruit and hire candidates, 
later discovering that a particular in- 
dividual may not be the best person for 
the position. Since recruitment that ends 
up like this is very costly, organizations 
are looking for methods of assessing the 
true skills of potential employees. At 
Walden Personal Testing & Training 
(http://www.waldentesting.com/), we pro- 
vide tools that help in the selection pro- 
cess. The tests we provide (which are 
typically licensed to and administered by 
an organization’s human resource de- 
partment) are not meant to replace the 
interview process, but are designed to 
be used in conjunction with in-depth in- 


Bruce is vice president of consulting ser- 
vices for Walden Personnel Testing and 
Training Inc. He can be contacted at tests@ 
waldentesting.com. 
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terviews and detailed reference check- 
ing. Walden tests are neither I.Q. nor 
personality based. Generally, they are 
not multiple choice, but instead simulate 
what the job applicant is required to do 
on the job, as well as to determine who 


is likely to be successful in technical 
training courses. 

Initially, our most popular test was the 
“Aptitude Assessment Battery: Program- 
ming” (AABP) test, which was designed 
in the late 1960s by Dr. Jack Wolfe (a pi- 
oneer of early data processing and com- 
puter education) to evaluate computer 
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programming potential. The five hour 
test is comprised of five very challeng- 
ing problems. It has been, and contin- 
ues to be, a successful evaluation tool. 
It not only answers the question, “Should 
you work as a programmer?” but also 
“How proficient will you be?” in a defini- 
tive manner. 

In 1983, we recognized there was a need 
to evaluate not only computer program- 
ming potential, but analytical skills as well. 
The ability to solve business-related prob- 
lems and create, interpret, and dissect de- 
tailed programming specifications is an in- 
tegral part of every developer’s job. 
Programmers are no longer just coders, 
they’re also expected to analyze detailed 
business requirements; hence, the job ti- 
tle “programmer analyst.” This led to the 
design and implementation of a new 
test— the Programmer Analyst Aptitude 
Test (PAAT). 

In developing PAAT, we made several 
design decisions, including: 


e The test should evaluate aptitude for 
the work. 

e The test should not exceed two hours. 

e The test should incorporate several of 
the skills and abilities evaluated in ear- 
lier programming tests. 

e At least 40 percent of the test should 
evaluate the candidate’s analytical skills. 
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(continued from page SO) 

e All questions should simulate pro- 
grammer analyst job-related functions. 

e The test must be designed so that the 
scoring is objective and suitable for fu- 
ture automation. 

e The test must be field-tested and vali- 
dated (content and predictive validation) 
to meet federal guidelines for fairness. 

e The test must be suitable for both entry- 
level and experienced candidates and 
assume no prior experience or knowl- 
edge of information technology. 


We have since added a one-hour ver- 
sion of the test that’s available over the 
Internet, and is administered in booklet 
form as well. 


How the Tests Were Developed 

My first step in developing the PAAT, was 
to conduct a thorough job analysis of the 
programmer/analyst position. I started by 
soliciting current job descriptions from sev- 






ity and attention to detail are extremely 

important. In this question, you will 
be required to understand the symbol- 
ic coding of instructions necessary to 
logically direct the “Starfire” spaceship 
to its various destinations. 

Rules for creating instructions from 
the coding symbols: 


n analyzing problems, your logic abil- 








1. Every instruction must start with an S. 

2. The next part of the instruction must 
be F, G, H, or I if Starfire is leaving 
Earth, or the number corresponding 
to the planet Starfire is departing from 
if it is not Earth. 

3. The last part of every instruction must 
be the number of the planet destina- 

tion plus “stop.” However, if the code 
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PAAT Sample Question 





SATURN 





eral client companies. It was apparent that 
approximately 10 job functions (tasks) 


Programmers are 
no longer just 
coders, they’re also 
expected to analyze 
detailed business 
requirements 





were common to most of the descriptions 
received. Next, I identified essential 
knowledge, skills, and abilities (KSAs) 


for the planet destination is an even 
number, it must be doubled before 
the “stop.” Example: 32E for Venus 
destination. 

4. The codes making up the middle of 
each instruction (between rules 2 and 
3 above) are 1,2,3, and 4 in a com- 
bination necessary for Starfire to trav- 

el along the proper path. 


Using our new “SYMBOLS,” the in- 
struction for sending Starfire from Earth 
(from the most northern space station) 
to Jupiter would be: SF131328E. (Please 
note that there is not enough informa- 
tion for you to actually solve this sam- 
ple question.) 


—B.W. 








JUPITER 






NORTH 


WEST 
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for each of the job tasks. The challenge 
here was to incorporate those KSAs that 
were most often identified, incorporate 
several others that were tagged sec- 
ondary traits, avoid multiple-choice ques- 
tions, and limit the exam to two hours. 
We then decided that questions would 
be developed around the following six 
high-level abilities: 


e Follow a complete procedure involving 
multiple transactions. 

e Create a routine procedure that solves 
a problem through the structuring of 
tasks. 

e Interpret detailed specifications, follow 
rules, and incorporate symbols to solve 
problems. 

e Analyze and execute a new instruction 
set, perform table look-up, and manip- 
ulate data to generate a solution. 

e Analyze a complex business situation 
and search through voluminous data to 
understand and identify core require-_ 
ments. 

¢ Create a symbolic program to solve a 
business-related problem and then iden- 
tify errors based on generated code. 


Ultimately, we decided that the first 
two problems should be developed 
around common banking procedures, be- 
cause many different types of transac- 
tions could be involved in the questions 
(deposits, withdrawals, and so on); the 
scenarios were common to most laymen, 
and business situations were involved. 
The actual creation of the questions then 
became easier, because all that had to be 
done was to model a realistic situation. 
Again, the trick was to limit both prob- 
lems to approximately 25 minutes for the 
candidate and to ensure this represented 
more than sufficient time to solve the 
problems. 

The third problem was developed 
around the general structure of comput- 
er machine language instructions (con- 
cepts of operation codes, data length, ad- 
dress, and so on). These concepts were 
made candidate friendly but required the 
ultimate test-taker to evaluate several 
complex expressions of increasing diffi- 
culty— all based on symbolic language 
architecture. 

The fourth problem extended the con- 
cepts the candidate applied in problem 
three, but forced the individual to per- 
form complicated table look-up proce- 
dures and then manipulate the data 
found to solve progressively difficult 
mathematical expressions. The creation 
of the question itself had as its founda- 
tion basic knowledge of tables and ar- 
rays, which are incorporated into most 
programming languages. The only chal- 
lenge was to translate this theory into 


http://www.ddj.com 


Developer Net. 





When you develop to Novell Directory Servicese (NDS™) and NetWarec 5, 
you turn the tables on your future. 





Employers know the global Internet economy is the future of business. If their applications don’t play well 
with the e-commerce directory of choice, their applications won't play at all. That's why they're looking for 


sawvy graduates who can develop to NDS—te cross-platform directory enabling the future of e-business. 


Which means NDS development should be on your resume—atter all, close to 50 million companies 
already use it, and it’s employed by 80 percent of the Fortune 500. And if analysts are half-right, by the 
year 2000, directory services product revenues will hit five billion—that’s five times 1998 revenues. 


What does that mean to the NDS developer class of 2000? Opportunity is going to beat your door down. 


Join Novell’s DeveloperNete Academic Program and get NDS development skills through 


on-line training with Novell DeveloperNet University. Sign up for your future at 


9 Novell, Inc. All rights reserved. Novell, NetWare, Novell Directory Services and DeveloperNet are 
ered trademarks, and NDS is a trademark of Novell, Inc. in the United States and other countries. All 
products and services mentioned are property of their respective holders. 














(continued from page S8) 

practice for an inexperienced candidate 
and restrict the time frame to about 20 
minutes. 

The fifth problem required detailing a 
complex business procedure in narrative 
form. It was modeled after a complicat- 
ed billing system for a fictitious compa- 
ny and forced the candidate to analyze 
the specifications in order to correctly 
select options and calculate invoice to- 
tals. This was the only problem that in- 
corporated multiple-choice questions and 
was done simply to minimize total test- 
ing time yet maximize the number of es- 
sential abilities evaluated. 

The sixth and final problem actually 
had the candidate write a symbolic pro- 
gram based on an instruction set the in- 
dividual learns from specifications pro- 
vided at the beginning of the problem. 
The individual was also provided with 
coding in this fictitious language and 
must detect errors in the logic. Once 
again, the design of the question itself 
was based on concepts of operation 
codes and addresses and was built 
around a common business procedure — 
adding payments. The difficulty was en- 
suring that the entire problem could be 
completed within 30 minutes. 

As a general rule of thumb, test prob- 
lems were organized from easier to more 
difficult in the final version of PAAT. Table 
1 lists the key traits evaluated by PAAT 
with an indication in which of the six 
questions they were contained. 


Sample Question 
The text box entitled “A Sample Question” 
presents an example of what a PAAT ques- 
tion might look like. You should be aware 
that there is insufficient data supplied in 
the problem to actually solve it. For rea- 
sons of security, we did not supply an ac- 
tual question from the PAAT. 

Based on the most recent candidates 
who have taken PAAT, the following statis- 
tics have been determined: 


Population Tested 5322.00 
Median (Mid- point) 74.00 
Minimum 0.00 
Maximum 100.00 
Mean (Average) 64.64 
Sample Standard Deviation 36.60 


Testing and the Law 

Antidiscrimination laws have been in effect 
in the United States since the passage of 
the Civil Rights Acts of 1866 and 1870, and 
the ratification of the 14th Amendment to 
the Constitution. More recently, there have 
been several acts and executive orders in- 
stituted to prohibit discrimination in em- 
ployment. One of the most important of 
these laws was the Civil Rights Act of 1964. 
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_ Prime Traits 


Ability to think logically. 
Ability to follow procedural logic. 


stated definitions. 
complex relationships. 


of specific cases. 


Se ee er ee 


trail and error alone. 
Ability to pay attention to details. 


Ability to adjust to changing situations. 


Ability to sustain concentration. 


Ability to correct coded information. 
Ability to produce original ideas in solving problems. 
Ability to analyze charts, tables and graphs. 


Ability to persevere. 
Ability to reason abstractly. 


Ability to attack a problem from many angles. 
Ability to translate business specifications into 


symbolic logic. 


Ability to remember details/instructions. 


Ability to handle new problems. 


Ability to solve problems utilizing reasoning 


with symbols. 

Ability to read. 

Ability to work independently. 
Ability to work under pressure. 
Ability to take corrective action. 
Ability to work at repetitive tasks. 
Ability to spot errors. 


Ability to comprehend written material. 
Ability to remember plans, rules and overall policy. 


Ability to show endurance. 
Ability to create procedural logic. 
Ability to learn. 

Ability to comprehend quickly. 


Ability to accurately check coded information. 


Ability to think critically. 


Ability to make decisions quickly and accurately. 


Ability to show patience. 


Ability to sort data in a prescribed sequence. 
Ability to follow complex procedures. 
Ability to utilize program logic development 


techniques. 


Ability to interpret intricate business specifications. 


Ability to do a complete job. 


Ability to analyze complex business procedures. 


Table 1: PAAT Prime Traits. 


Title VII of the Act prohibits discrimi- 
nation because of race, color, religion, sex, 
or national origin, in all employment prac- 
tices, including hiring, firing, promotion- 
al, compensation and other terms, privi- 
leges, and conditions of employment. 

The U.S. Equal Employment Opportu- 
nity Commission (EEOC) was created to 
monitor companies and assure adherence 
to Title VII. The Equal Employment Act 
of 1972 provided EEOC with direct ac- 
cess to the courts and subsequently en- 
hanced the Commission’s effectiveness 
as a compliance-enforcing agency. 

At first, the EEOC focused its efforts on 
the more blatant element of discrimina- 
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Ability to accurately solve problems 
Ability to reason with symbols according to 


Ability to perform basic arithmetic calculations. 
Ability to understand and observe highly 


Ability to work rapidly and accurately. 
Ability to deduce generalizations from a series 


Ability to follow instructions precisely. 
Ability to analyze a problem not solvable by 
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tion. Then the Commission addressed the 
application and use of employee selection 
procedures. The results of its actions were 
a set of standards for employee selections. 
These standards evolved into the Uniform 
Guidelines of Employee Selection Proce- 
dures. 

The Uniform Guidelines are a set of 
standards to be used in determining 
whether employee section procedures are 
unlawful. The Guidelines were developed 
to be consistent with legal standards and 
validation standards accepted by the psy- 
chological profession. 

The Guidelines apply to any procedure 
used to make an employment decision. 
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Dr. Dobb's/CD Release 7 brings 
together more than eleven years 
worth of perspectives and expertise from the most respected program- 
mers in the industry. Since Dr. Dobb's Journal is written by program- 
mers, for programmers you receive the news and views from inside the indus- 
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Patterns and Software Design, Software 
Debugging and Testing, File Formats, Embedded 
Systems, Database Development, Portability and 
Cross-Platform Development, and Web 
Development. 
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This includes oral and written tests, in- 
terviews, review of application blanks, 
review of work experience, work sam- 
ples, and physical requirements. 

The underlying principle is that the em- 
ployer policies or practices having adverse 
impact on employment opportunities of 
any race, sex, or ethnic group are illegal 
under Title VII, unless justified by business 
necessity. A selection procedure found to 
have no adverse impact generally does not 
violate Title VII. Adverse impact occurs 
when a selection procedure results in dif- 
ferential rates of selection or rejection of 
the various minority and gender groups. 

This means that if an employer is using 
a selection procedure that does not have 
an adverse impact on a protected group, 
the employer may avoid the application of 
the Guidelines. If adverse impact exists, the 
Guidelines state that it must be justified on 
grounds of business necessity, that is, there 
is no other method of evaluating job suit- 
ability. 

According to the Guidelines, validation 
demonstrates the job-relatedness of a se- 
lection procedure. For example, if a se- 
lection procedure is a test, the employer 
must show that the test scores are related 
to performance of the job for which the 
employees are being selected. 

The Guidelines recognize three types 
of validation strategies: 


¢ Criterion-related validity, which calls for 
the demonstration of a statistical rela- 
tionship between selection- procedure 
scores (predictors) and the job perfor- 
mance of workers (criterion). 

e Content validity, which calls for an in- 
vestigation of the degree to which the 
test includes a representative sample of 
all the tasks that make up the job. The 
content of the selection procedure must 
be representative of important aspects 
of job performance. 

¢ Construct validity, which calls for the 
demonstration that a selection proce- 
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dure measures a specified attribute (con- 
struct), and that the selection procedure 
can be used to predict job performance. 
Construct validity requires a great deal 
of empirical data, and usually requires 
more than one study. 
Of the three types of validation mod- 
els accepted by the Guidelines, criterion 


related and content validity are the most 
practical and widely used. 


We wanted to 
demonstrate that the 
test would 
predict the overall 
programming/ 
analytical potential 
of an applicant 





Content Validity of PAAT 

With regard to establishing content valid- 
ity, all of the major essential functions (ac- 
tivities) required for the programmer an- 
alyst job position are identified based on 
data supplied by BCP Bank Card Pro- 
cessing Worldwide via a recent job de- 
scription (1997). A copy of the job de- 
scription for the programmer analyst is 
collected, verified as being up-to-date, 
and filed for future reference. 

Each of the identified essential functions 
is ranked (weighted) in order of impor- 
tance to the programmer analyst position 
by BCP Bank Card Processing Worldwide 
personnel. 
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Next, the traits, skills, or knowledge re- 
quired to perform these functions are iden- 
tified with an indication as to whether 
those essential traits are evaluated by a 
relevant PAAT question or some other 
means (such as on-the-job observation). 

Also, the major secondary traits (nice 
to have, not essential) are identified for 
each of the key functions performed. It is 
noted whether those secondary traits are 
evaluated by a relevant PAAT test ques- 
tion or some other means. The goal is to 
ensure that all the major essential func- 
tions are specifically and accurately de- 
fined and categorized into behavior do- 
mains that can be evaluated. 

For purposes of the programmer ana- 
lyst, a complete job analysis was con- 
ducted. Fourteen key functions are iden- 
tified for the job position. It is these 14 
essential functions for which the PAAT is 
being used for evaluation. 


Results of the Job Analysis Procedure 
As the job analysis indicated, the pro- 
grammer/analyst had to perform 14 es- 
sential functions to successfully execute 
the responsibilities associated with the job. 

A total of 56 traits were judged to be 
essential to accomplishing the 14 es- 
sential functions. Of these essential traits, 
360 were assessed by one or more sec- 
tions of the test. Thus, a substantial por- 
tion (64.3 percent) of the intended do- 
main was assessed by the test. In 
addition, 22 secondary traits (defined as 
those that are “nice but not necessary”) 
were identified for the programmer an- 
alyst—Introductory/Trainee job posi- 
tion, of which five were tested, but not 
quantified. If the relative importance of 
the essential traits tested to the weight- 
ed importance of the job functions are 
considered, the overlap percentage im- 
proves to 69.3 percent. 

Given the demonstrated relationship 
between the abilities and traits required 
to perform the 14 essential functions of 
the programmer analyst job, and those 
measured by the PAAT, the test repre- 
sented a content-valid evaluation device 
for that job. 


Predictive Validation 

To conduct a predictive validation study 
of the Walden Programmer Analyst Apti- 
tude Test, we wanted to demonstrate that 
the test would predict the overall pro- 
gramming and analytical potential of an 
applicant on the job as measured by su- 
pervisory ratings, for future employees in 
a programmer analyst job position. 

In addition to the content validity de- 
scribed previously, BCP supplied us per- 
formance ratings of 63 applications pro- 
grammers. The intent was to determine if 
a predictive relationship existed between 
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success on-the-job and high scores of the 
PAAT. These scores were compared using 
the SPSS statistical software package. 

There exists an empirical relationship, 
(r=0.457), which is significant to the 0.01 
level, between high scores on the PAAT 
and programmer performance on-the-job. 
ris known as the “correlation coefficient” 
(or the “Pearson r”). It is used to indicate 
the strength (or lack thereof) of the cor- 
relation between two variables. On the 
PAAT, the conclusion was quite clear: The 
higher the candidate’s score on the PAAT, 
the greater likelihood that the candidate 
would be rated higher on the job, based 
on performance. At the 0.01 significance 
level, you could expect this conclusion to 
have occurred by chance only one time 
out of 100. The legally accepted range for 
levels of significance is between 0.05 and 
0.01. Given the empirical relationship be- 
tween higher performance rankings and 
higher scores on the PAAT, the test rep- 
resents a predictive valid selection device 
for this job. 

In addition (based on 63 hired candi- 
dates): 


e The mean score for the PAAT was 82.2 
percent. 

e The median score for the PAAT was 90.0 
percent. 

e The mode score for the PAAT was 94.0 
percent. 

e The standard deviation for the PAAT was 
12.4 percent. 

¢ The range for the PAAT was 60 points. 

e The minimum score for the PAAT was 
40 percent. 

e The maximum score for the PAAT was 
100 percent. 

e The mean score for males was 86.2 per- 
cent with a standard deviation of 11.8 
percent. 

e The mean score for females was 83.2 
percent with a standard deviation of 13.6 
percent. 

e There was a significant empirical rela- 
tionship for both males and females be- 
tween high scores on the PAAT and high 
performance ratings on the job. 


Conclusion 

From a practical point of view, the PAAT 
definitely identifies those people with a 
high aptitude for programming and for 
systems analysis. It also identifies those 
individuals who are high-risk hires dow 
scores). This is the true value of the test. 
It minimizes the costs associated with a 
poor employment decision. A more recent 
development has been that an Internet ver- 
sion of a similar test is now available for 
clients who need an immediate response 
as to candidate suitability. 
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Matching your 
values to those of 
prospective employers 





Greg Scileppi 


erome Freeman was elated when he 
was Offered a job as an applications 
developer at a promising startup that 
designs web sites devoted to e- 
commerce. But a month into the job, Free- 
man, a disciplined and independent self- 
starter, realized that the company wasn’t 
meeting his expectations. While coworkers 
routinely worked 11-hour days, much of 
that time was spent in personal discussions 
in the coffee room while deadlines loomed. 
Meanwhile, Freeman was spending more 
and more time attending day-long meetings 
that consisted primarily of colleagues at- 
tempting to ingratiate themselves with se- 
nior management. Lacking a taste for office 
politics, he found these sessions even more 
frustrating since they left little time for him 
to spend actually designing applications. 


Greg is executive director of RHI Con- 


sulting. He can be contacted at gscileppi@ 
rhic.com. 
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Culture 


It soon became apparent to Freeman 
that he had made a mistake in accept- 
ing the job. Worse, he wasn’t precisely 
sure how he had so completely misread 


SS ee = 












ites, 


aes 


what it would be like to work for the 
firm. Reluctantly, he began to look for 
another job. 

While Freeman’s needs and preferences 
may not coincide with your own, they are 
reflective of concerns shared by a growing 
number of information technology profes- 
sionals today as they assess their career op- 
tions. A decade or two ago, technology pro- 
fessionals might have based their decision 
to sign on with a firm almost entirely on 
salary. But today, employees at all levels 
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recognize that job satisfaction involves more 
than just a paycheck and an attractive ben- 
efits package. They realize their ultimate 
job satisfaction is also dependent on the 
unique set of values and attitudes displayed, 
sometimes subtly, sometimes not, by their 
employer. And, increasingly, they’re able to 
get what they want. 

Today’s IT job candidates are in the 
driver's seat. And one of the factors they’re 
using to evaluate a prospective employer 
is the organization’s corporate values. The 
challenge, as Freeman learned firsthand, 
lies in accurately “reading” a firm and its 
work environment prior to making a full- 
time commitment. 

How do you make sure your values 
are closely aligned with the personality 
of your future employer? You can start 
by recognizing that there are many as- 
pects to a company’s culture. Just as an 
individual has a distinct personality he 
or she shows to the world, every business 
is imbued with its own unique set of val- 
ues. For true job satisfaction, your task 
is to determine which values are most 
important to you and then find the firm 
that offers the closest match. 


Work/Life Balance 

Number One for Many 

Of all the corporate values held by USS. 
businesses across all industries, probably 
the most widely publicized and most of- 
ten cited by employees as important to 
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(continued from page S16) 

them is a concern for workers’ ability to 
reconcile the demands of their jobs with 
their personal lives. 

Accommodations for telecommuting, flex- 
ible schedules, and dependent care are in- 
creasingly important as workers try to 
achieve a satisfying 
work/life balance, 
and a growing num- 
ber of firms are now 
offering these bene- 
fits. A North Carolina 
manufacturer I work 
with, for example, ar- 
ranges for school 
counselors to meet 
on-site with parents 
during regular work- 
ing hours and gives 
employees time off to 
attend special events 
at the company’s day 
care center. In a sim- 
ilar vein, an Oregon-based computer com- 
pany recently extended its day care to in- 
clude kindergarten and first grade. 


Intellectual Challenge 

Key to Job Satisfaction 

Most IT professionals are aware of the need 
to keep their software and hardware skills 
current to maintain their marketability. In 
fact, a number of consultants come to me 
seeking contract work rather than full-time 
employment because the diverse assign- 
ments enable them to stay on top of a 
rapidly evolving industry. Technology pro- 
fessionals, in particular, view continuing 
education as a prerequisite to their career 
survival, an observation that was confirmed 
by a recent study conducted by RHI Con- 
sulting (the company I work for). When 
ClOs participating in the study were asked 
to choose the single factor, other than com- 
pensation, that helped managers retain IT 
talent, they placed ongoing technical skills 
training at the top of the list. 

If learning is a priority for you, look for 
organizations that encourage their man- 
agers to solicit new ideas from employees 
at all levels. Firms that value intellectual 
growth are typically supportive of people 
who experiment with different approach- 
es, and they tend to reward prudent risk 
taking. They invest generously in their staff 
by providing opportunities to learn on the 
job, including company mentoring pro- 
grams and tuition reimbursement for out- 
side training. Is it important for your future 
employer to offer you the chance to ex- 
pand your skills and the freedom to move 
from project to project? 


Effective Leadership 


Most people would agree that the most 
effective leaders have a talent for build- 


Dr. Dobb’s Journal, November 1999 


Accommodations for 
telecommuting, 
flexible schedules, 
and dependent care 
are increasingly 
Important 








ing enthusiasm for an organization’s goals 
and inspiring employees to put forth their 
best efforts. There is far less agreement, 
however, on what leadership means to 
each individual employee. 

Some favor clear direction in their 
work, seeking well-defined, unambigu- 
ous tasks, for example, 
while others find this 
structure confining and 
prefer more autonomy. 
Then again, some em- 
ployees may view too 
much autonomy as an 
abdication of leadership 
responsibilities. 

Yet another group of 
employees, albeit a 
smaller group in today’s 
fast-paced business en- 
vironment, finds comfort 
in a traditional, hierar- 
chical management 
model in which each 
position is defined explicitly. Their coun- 
terparts, on the other hand, look for lead- 
ers who will serve more as coaches than 
bosses, favoring project- driven collabora- 
tion over a chain of command. It’s im- 
portant to determine in which environ- 
ment you feel most comfortable. In his 
case, Jerome Freeman realized too late 
that the culture at his new job was not a 
match for his personality and work ethic. 

As you evaluate a prospective employ- 
er, consider its leadership. What are the 
characteristics of those in senior manage- 
ment? Try to identify the kind of behavior 
that is typically rewarded. A company’s 
compensation program can also help you 
identify the predominant management 
style, with some companies tying bonus- 
es to the success of departments or teams, 
and others basing them primarily on in- 
dividual contributions. 


Making a Difference 

Is it meaningful to you for your firm to 
be seen as a good corporate citizen in 
the communities in which it does busi- 
ness? Grant-making programs and sup- 
port for employee volunteer activities are 
powerful magnets for those who want 
to be associated with a company that is 
an active contributor to causes impor- 
tant to them. Such policies can also ex- 
tend the reach of individual employees 
who want to support the work of spe- 
cific organizations or to call attention to 
socially relevant issues. Employees at one 
Northern California network systems 
firm, for instance, have formed a special 
company-supported committee to orga- 
nize activities such as weekend cleanups 
of local creeks and beaches, video con- 
ferences on environmental issues, and fund 
raising for environmental organizations. 
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Likewise, one Midwest clothing retail- 
er offers employees up to 35 hours of paid 
time off per year to engage in communi- 
ty activities related to education or health 
and social services. Do any of these types 
of support reflect values you’d like your 
future employer to hold? 


Company Culture 

There are a number of ways to get an ac- 
curate feel for a business’s values prior to 
signing on. While the job interview is the 
most obvious, there are steps you can take 
both before and after the interview to as- 
sess the underlying philosophy driving the 
policies of a potential employer. 

For starters, the Internet is full of in- 
formation about both large and small com- 
panies. Conduct a search to learn as much 
as you can about the firm before the in- 
terview, particularly as it relates to its busi- 
ness goals and industry reputation. The 
look and content of its web site can also 
give you a sense of the company’s values 
and prevailing culture. Is the site cutting- 
edge and animated or subdued and so- 
phisticated? This speaks volumes about 
the image they wish to convey to the 
business community and the public. 

If you’re called in for an interview, ar- 
rive a few minutes early so that you can 
observe how employees dress, how they 


interact, and what the physical workspace 
is like. Sometimes you'll have to look past 
the superficial to properly interpret the 
real messages an organization is sending. 
An offering of substantial on-premises ser- 
vices could reflect either a company ea- 
ger to please its employees, or a corporate 
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culture focused on maximizing the time 
employees spend on the job. One Silicon 
Valley firm, for example, provides such 
on-site services as dry cleaning, film de- 
veloping, and dinners-to- go. 

To get a further glimpse into an orga- 
nization’s values, speak with current and 
former employees. Professional trade as- 





sociation meetings can also provide an 
opportunity to connect with others who 
have knowledge of the company in which 
you are interested. In addition, you may 
want to speak with recruiting or consult- 
ing firms, which typically have strong ties 
within the business community and are 
familiar with the work environment and 
culture at local firms. If you’re not suc- 
cessful obtaining enough information from 
these sources to make your decision, con- 
sider working as a contract or temporary 
employee before committing yourself to 
a full-time position. 


A Winning Hand 

You never really know a city until you’ve 
lived there. You never really know a neigh- 
borhood until you’ve moved in. So too 
with an employer. It’s impossible to an- 
ticipate all of the subtle attitudes and poli- 
cies that contribute to a company’s cul- 
ture. At the same time, by seeking to 
understand the firm’s core values— then 
comparing them to your own— you can, 
at the very least, curtail any surprises and, 
at most, find yourself working for a com- 
pany whose work environment and man- 
agement style are ideally suited to your 
own personal and professional ambitions. 
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The future looks bright 
for problem solvers 





Barbara Dijker 


hat are system administrators? In 

general, sysadmins are those peo- 

ple we complain to when our 

computer systems aren’t working 
the way we expect. If they can make ev- 
erything right, then they must be system 
administrators. Therefore, a system ad- 
ministrator is someone who solves prob- 
lems in computer and network systems 
operations. 

The problem set in computing and net- 
work operations generally includes all 
those system tasks users might want to 
offload — specification, evaluation, instal- 
lation, configuration, integration, mainte- 
nance, data-integrity management, upgrade 
management, automation, security man- 
agement, performance analysis, failure 
analysis, failure mitigation, recovery de- 
sign, recovery implementation, testing, and 
more. In reality, the definition of system 
administration is usually a subset of all 
those tasks specific to the organization do- 
ing the hiring. For example, some orga- 
nizations have a separate staff of network 
administrators to handle network admin- 
istration, while in other organizations sys- 
tem administrators manage both comput- 
ers and network infrastructure. Other 
organizations have separate staff dedicat- 
ed to security administration. Some orga- 


Barbara is vice president of the System Ad- 
ministrators’ Guild (SAGE) and has been 
a system administrator for 13 years, work- 
ing in aerospace, higher education, and 
the commercial sector. She can be con- 
tacted at barb@netrack.net. 
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nizations allow users a high degree of self- 
administration that makes the system ad- 
ministrator’s job focus more on the inte- 
gration and systemic issues. 


Specialties and Career 

Advancement Paths 

Because system administration is so broad, 
room exists for specialization. Examples of 
where you can specialize are security, data 
encryption, networks, heterogeneous inte- 
gration, specific operating systems or plat- 
forms (Sun, NT, or Linux), specific subsys- 
tems (file sharing, mail, or naming systems), 





specific applications (supercomputing), and 
(more recently) web-server administration. 
Web-server administrators specialize in the 
operational issues specific to web servers, 
usually in the area of performance, securi- 
ty, reliability, and redundancy. 

System administration is often thought 
of as a job in the trenches. However, good 
system administrators, through experience, 
hone unique problem solving skills, peo- 
ple skills, organizational skills, and a keen 
sense of the big picture and potential im- 
plications of change. These skills are in- 
valuable for someone interested in a man- 
agement track that can lead up the chain 
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Administration 


all the way to corporate information tech- 
nology director and even chief technical 
officer. 

Because system administrators seem to 
enjoy the challenge of actually fixing 
problems, career advancement often tends 
to stay on a more technical track. System 
administrators have plenty of responsi- 
bility for the operational integrity of the 
systems they administer, so the lure of 
responsibility for staff or budgets is less 
appealing. Advancement technically can 
be toward: 


e Consulting, which often provides the 
ability to select work or truly focus on 
a specialty. 

e Back-line, away from the “front-line” of 
interruptions to the more challenging 
problems. 

e Specialties, focused and more detailed 
work, more available in larger organi- 
zations. 


Large organizations may provide pre- 
defined advancement from the front-line 
of basic user support or “help-desk” 
through multiple levels of system admin- 
istration toward the back-line of elite trou- 
bleshooters. Some companies even have 
a “special forces” team of system admin- 
istrators to swoop in and fix really tough 
problems. 


Market Areas and Industries 

Every organization that has computing sys- 
tems needs someone to perform system 
administration tasks. While every such site 
may have that need, solving it does not 
always involve hiring a full-time system 
administrator. When the load of system 
administration tasks is low, they are usu- 
ally delegated to an existing staff person 
whose primary responsibility is something 
else. Interestingly, this is often the gene- 
sis of a system administrator. That person 
finds they enjoy the system administration 
tasks more than his other work. 
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(continued from page S20) 

The affect of this on the market for sys- 
tem administrators is that you can find a 
job in system administration in any in- 
dustry in any market sector. Small, grow- 
ing, and changing organizations feed the 
free-lance consulting and contract system 
administrator markets. Other organizations 
feed the consulting and contract pools 
with experienced system administrators 
who are looking for a change. In short, 
system administrators work for everything 
from churches and hospitals to multi- 
national and Fortune 500 companies. 


Market Demands and Trends 

Some people have long predicted that the 
need for system administration would die 
out like dinosaurs as computers and ap- 
plications become more automated and 
sophisticated and solve their own prob- 
lems. The opposite is actually the case. 
As computers and applications have be- 
come more sophisticated, they’ve created 
different problems from those they may 
have solved. For example, ubiquitous com- 
puter networks have solved many prob- 
lems in transport of information and re- 
source sharing. However, for each 
problem solved, networking has intro- 
duced at least a dozen in the areas of se- 
curity and data integrity. As problems crop 
up, system administrators have to figure 
out what to do about them to allow users 
to continue the “real” work. 

There has never been a market glut of 
systems administrators. In fact, there are 
probably five jobs available for every ex- 
isting sysadmin today. In fact, most ex- 
perienced system administrators find they 
are doing the work of five people. De- 
mand has out-paced supply for at least 
the last six years. 

Of course, this is mostly subjective in- 
formation. There are at least two problems 
with clearly quantifying trends in the sys- 
tem administrator job market. The first 
problem is that most system administrators 
have a job title that has nothing to do with 
system administration — a member of tech- 
nical staff, systems programmer, systems 
analyst, and so on. According to studies 
conducted by the Systems Administrators’ 
Guild (SAGE; http://www.usenix.org/sage/ 
jobs/salary_survey/index.html), fewer than 
half of all system administrators are giv- 
en title recognition for it. The second prob- 
lem is that market analysts don’t usually 
track system administrator as a unique job 
title because of the first problem. Any sur- 
veys of system administrators are based 
on self-identification, not job title. 


Salaries and Salary Trends 

As you would expect with increasing de- 
mand and shortage of qualified appli- 
cants, salaries for system administrators 
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are attractive and competitive. Accord- 
ing to SAGE studies, the average salary 
for all system administrators — regard- 
less of experience, industry, or geogra- 
phy—was approximately $62,000 in 
1998. System administrators with 1-2 
years of experience on average earned 
$47,000 while those with 10 or more 
years of experience earned more than 
$70,000. About five percent of all sysad- 
mins earned over $100,000 in 1998. As 


Every 
organization that 
has computing 
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perform system 
administration tasks 


with most jobs, salaries are higher in the 
commercial sector than they are in edu- 
cation, government, and other organiza- 
tions. Salaries are also higher in parts of 
the country where there is more demand 
and higher costs of living, such as Cali- 
fornia and New England. 

Salary increases for system adminis- 
trators tend to be larger than those in oth- 
er jobs, usually to keep someone from 
leaving for greener pastures. The increase 
in average salary from 1997 to 1998 was 
6 percent. Salary disparities between men 
and women are significantly less in sys- 
tem administration than other fields. In 
fact, the 1998 survey referenced here in- 
dicates women made about 15 percent 
more than men. However, that could be 
attributed to more men coming into the 
field than women. System administration 
tends to attract more women than other 
technical and computing fields. Sixteen 
percent of system administrators are 
women. 


Consulting and Contracting 
Versus Direct Employment 
Because of the large demand for system 
administrators, there is also a large mar- 
ket for “on-demand” system administra- 
tors through contract employment or con- 
sulting. Some system administrators prefer 
consulting or contracting because it pro- 
vides greater variety and flexibility. 
System administrators who are inde- 
pendent consultants tend to be specialists 
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and make significantly more money than 
those who work for a firm and are con- 
tracted out. (For more information, see 
the SANS Salary Survey, http://www.sans 
.org/newlook/publications/1998salary- 
survey.htm). As with most other comput- 
ing contract employment, those who work 
for contract firms tend to make slightly 
more money than those who are directly 
employed. In this respect, system admin- 
istration is similar to other technical fields. 
The only difference may be that due to 
significant demands for system adminis- 
trators, making a go of it as an indepen- 
dent may be more readily lucrative than 
in other fields. There are plenty of op- 
portunities regardless of which employ- 
ment structure is most attractive. 


Job Requirements 

Becoming a system administrator is rela- 
tively straightforward. Anyone curious to 
explore a challenge while possessing a 
reasonably solid background in comput- 
ing fundamentals may find themselves 
drafted anyway. Due to demand, organi- 
zations are also more likely to consider 
offering entry-level positions with train- 
ing than in other computing fields. 

Gaining a background in computing 
fundamentals is pretty easy. A degree in 
computer science isn’t required, just a ba- 
sic understanding of how computers, op- 
erating systems, and applications work 
and interact, logic, binary numbers, a lit- 
tle electronics, and some common sense. 
The most important traits of a new sys- 
tem administrator are the abilities to find 
and digest information, solve problems, 
develop creative strategies, and recognize 
and work within one’s own limits. System 
administrators who have formal education 
and a degree tend to be preferred and 
sometimes required for jobs. However, 
that’s not necessarily because the educa- 
tion is particularly relevant to the job, it’s 
more because it is evidence of follow 
through and perseverance. Many practic- 
ing system administrators hold degrees in 
technical fields other than computer sci- 
ence, such as physics. 

System administrators solve problems 
daily. The problems change every day and 
there is no step-by-step guide for solving 
them. So a system administrator has to be 
able to think on his/her feet, learn about 
the aspects of the problem, find the cause, 
find a fix, and then implement one that 
actually works within the constraints of 
the situation. Sysadmins are usually work- 
ing on production systems, because it is 
expensive to duplicate a system purely for 
the system administrator’s benefit. Conse- 
quently, it is imperative that a system ad- 
ministrator knows when he/she is in over 
his/her head— and how not to create 
more problems. Other than that, it’s all 
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(continued from page S22) 
about experience and system-specific — 
UNIX, NT, applications — training. 


Training and Education 

There is no widely accepted formal cur- 
riculum for system administration. Unfor- 
tunately you can’t go to the local univer- 
sity and sign up for a degree program in 
system administration. 

Still, there are plenty of education and 
training opportunities for system admin- 
istrators, including commercial education 
programs, university continuing educa- 
tion, vendor-specific courses, conference 
and seminar tutorials, and self-education 
through books. Commercial technical train- 
ing companies, such as Technology Ex- 
change Company (http://www.technology- 
exchange.com/) and Global Knowledge 
Network (http://www.globalknowledge 
.com/), also provide system-administration 
courses. Major platform vendors (Sun, 
Microsoft, Novell, and the like) provide 
system-administration training as well. 

Nonetheless, universities are starting to 
develop system-administration curricula. 
The University of Indiana, for instance, of- 
fers an independent study program in sys- 
tem administration called “UNIX System Ad- 
ministration Independent Learning” (USAIL; 
http://www.uwsg.indiana.edu/usail/). The 


System Administrators’ Guild (SAGE; 
http://www.usenix.org/sage/) provides 
publications, conferences, and tutorial pro- 
grams on system administration. The 
number of books on system administra- 
tion increases all the time. There are cur- 
rently about 75 books in print on various 
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aspects of system administration. Two 
books stand out as required reading for 
UNIX system administrators: UNIX Sys- 
tem Administration Handbook, by 
Nemeth, Hein, Snyder, and Seabass 
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(http://www.admin.com/) and Essential 
System Administration, by Frisch 
(http://www.ora.com/catalog/esa2/). 
Furthermore, there are certification 
programs for system administration. Most 
existing certification programs are ven- 
dor specific; such as Solaris and/or NT 
administration, for instance. The Linux 
Professional Institute (http://www.Ipi 
.org/) is developing certification for Lin- 
ux professionals, who are essentially sys- 
tem administrators. SAGE is conducting 
an occupational analysis of system ad- 
ministration to begin the process of the 
only platform-independent system ad- 
ministration certification program. 


Conclusion 

The profession of system administration 
and programs for system administrators 
are growing all the time. In part, to meet 
this demand, SAGE is currently conduct- 
ing a pilot program in mentoring and plans 
to have an established mentoring program 
next year. Furthermore, companies such 
as Sun are creating new multifaceted pro- 
grams to spawn and encourage new sys- 
tem administrators. Why? because system 
administration is a career that is in high 
demand— with no let up in sight. 
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PROGRAMMER’S TOOLCHEST 








e eXtended Markup Language (XML) 
provides great flexibility when it comes 
to data description and data structure. 
Applications need more than just data, 

however. Services are key application com- 

ponents. In this article, I'll discuss how XML, 

in combination with the Common Object 

Request Broker Architecture (CORBA), can 

provide an application-development envi- 

ronment that is flexible and rich in services. 

The key benefits CORBA provides are 
the separation of interface and imple- 
mentation and the abstraction of common 
application services. Services— security, 
management, transactions, and the like — 
are lifted out of the application logic and 
implemented in separate processes. XML, 
on the other hand, focuses on data de- 
scription and structure. XML is actively 
used in such areas as data interchange, 
cross-format document management, 
stream-based representation of complex 
objects, and semantic bridging. 

Although XML is not a replacement for 
either CORBA or COM+, some people still 
view it as infinitely flexible middleware. 
Consequently, there are a number of com- 
mon misconceptions about XML. 


e XML is not middleware. XML deals with 
the representation and structure of data. 
CORBA is a distributed object standard. 
Data representation is an important but 
small part of a distributed application. 
XML is not an ORB. This is mostly “mar- 
ketingware.” One XML-vendor, in fact, 
promotes its XML-based software as a 
replacement for an ORB. Upon close in- 
spection, however, they use XML in a 
role similar to Common Data Repre- 
sentation (CDR) in Internet Inter-ORB 
Protocol (IIOP). 
e XML and HTTP are not the same. A pro- 
tocol must fit the usage profile of an ap- 
plication. Some parts of an application 
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XML and CORBA 


The XML/IT toolkit 
combines the best of two worlds 


Dirk Hamstra 


may require a synchronous connection 
while other parts are better served using 
a queuing or multicast mechanism. Ide- 
ally, message delivery is based on mes- 
sage properties that are set and resolved 
at run time. XML can flow over any trans- 
port mechanism. This means that XML 
data can be shipped over protocols such 
as MQ or IIOP, in addition to HTTP. 
¢ XML does not handle all data types. All 
data in XML is represented as strings. 
This helps human readability, but re- 
duces performance for nonstring data 
types. In general, XML’s type system is 
primitive, which makes it useful on the 
Web, but not suitable for manipulating 
complex, typed data. Through the com- 
bination of Interface Development Lan- 
guage (IDL) and IIOP, CORBA provides 
a variety of data types (integer, string, 
octet, sequence, and the like). 
XML is not a better IDL. For all practical 
purposes, interfaces can be defined in 
any arbitrary language. Replacing IDL with 
XML is just a swap of languages. If you 
compare Listing One (the interface in 
XML) with Listing Two (the interface in 
IDL), for instance, you'll see that trading 
IDL for XML is not useful, since XML is 
not as compact and precise as IDL. 


Because CORBA and XML are comple- 
mentary technologies, there are a number 
of benefits you realize by combining the 
two technologies, such as: 


e Replacement of CDR. XML overlaps with 
CDR. Replacing CDR with XML allows 
for more flexibility in the representation 
of data above and beyond the data types 
currently available. XML can also reduce 
problems with IIOP versioning. 

e XML+IDL. Using XML and IDL in com- 
bination provides for the discovery of 
object structure and methods at run time, 
rather than generating compilable code. 
Benefits include economical, efficient, 
and extensible requests and responses 
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for dynamic object creation and method 
invocation. 

e Object serialization. XML is a good lan- 
guage to stream out objects. XML is 
close to a bytestream. Shipping flattened 
XML objects using a CORBA infrastruc- 
ture is a natural match. 

e Rich data. A CORBA call can return an 
XML document with, for example, mon- 
itoring statistics, including object refer- 
ences in the data. This allows more flex- 
ible and comprehensive interaction 
between software processes. 


CORBA+XML: An Example 
In general, using CORBA and XML to- 
gether means that the input and output of 
a CORBA method call is expressed in 
XML. To illustrate this process, I’ll use an 
XML formatting that’s based on the func- 
tionality of a Java tool called XML|IT from 
CareFlow (http://www.careflow.com/). 
XML|IT allows automatic tagging of the 
results returned from calls to CORBA- 
based services. It also includes utilities that 
support conversion of XML-tagged docu- 
ments to Java structures, and vice versa. 

In addition to assuming the existence of 
CORBA-based back-end services, XML | IT 
assumes the use of IDL-to-Java compilers 
to generate Java stubs and skeletons, and 
uses CORBA’s Dynamic Invocation Inter- 
face standards specification of CORBA. The 
main client program routine (D/[Call) can 
be embedded inside other Java-based 
clients, CGI scripts, or Java servlets. For ex- 
ample, to convert a Java structure to XML, 
you would call XML|IT’s JavaToXML from 
within Java using Listing Three. As a result, 
an XML representation of bankObj will be 
put in the String called xml. Mapping ar- 
rays to XML is handled through the use of 
a dimension attribute. Listing Four is the 
Java representation, while Listing Five is 
the converted XML representation. 

For the conversion to and from Java, 
XML|IT makes use of the Fascade Pat- 
tern combined with Java reflection. The 
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Figure 1: Generic view of a system that implements the accountList function. 


(continued from page 98) 

Interface com.careflow.xml.Accessor im- 
plements the Fascade Pattern. The Acces- 
sor provides a unified interface to elements 
that may have as a parent an Avray, a plain 
structure, or no object at all. The class 
com.careflow.xml.ReflectionTraverser iter- 
ates over all the elements of an Object by 
reflection. Each element is visited by the 
given implementation of Reflection Visitor. 
During a ReflectionTraversal, the current 
node may be a field in a structure, an ele- 
ment in an array, or the root element. The 
Accessor provides a common interface to 
these three cases, allowing them to be con- 
veniently handled by the Reflection Visitor 
and the ReflectionTraverser. 

To invoke a method on a CORBA ser- 
vice, the input XML document must be for- 
matted using some general guidelines. List- 
ing Six exemplifies this. The XML document 
can be divided into three sections. 


e The first part is contained between the 
<servicename> tags and describes the 
location and name of the CORBA ser- 
vice that will be used. Listing Six shows 
how to make a dynamic invocation. Stat- 
ic calls do not need to reference the im- 
plementation repository. 

e A section detailing the method invoca- 
tion follows the service specification. 

e The last part delineated by <returntype> 
specifies the data type containing the 
results of the invocation. 


Listing Six shows a CORBA method in- 
vocation expressed in XML, specifically 
the invocation of the method accountList 
on a service called Bank, located on ma- 
chine ChicoSpuugbeest. The function takes 
two parameters, customerld and dc- 
countNo. The result of the operation will 
be returned in a sequence called Cus- 
tomerAccountSequence. 


Figure 1 provides a generic view of a 
system that implements the accountList 
function. The function is implemented by 
a CORBA service called accountView. The 
associated IDL for the service can be ex- 
pressed as in Listing Seven. How the ac- 
countView service is implemented is hid- 
den from the requestor. Execution of the 
accountList operation may involve multi- 
ple transactions to different systems to 
build the list of accounts. Also, because 
account information can be sensitive, a 
security service may be used to authenti- 
cate and authorize the request. The trans- 
port medium for XML is also open. The 
request could come from a browser that 
passes the XML flow, or from a message 
queue to the CORBA service. 


Conclusion 

CORBA and XML are different, comple- 
mentary pieces of the same puzzle. CORBA 
provides a solid distributed framework 
with rich application services. XML gives 
you a flexible, dynamic data representa- 
tion and structure. XML is intended for 
the storage and manipulation of text com- 
prising human-readable documents such 
as web pages, while architectures such 
as CORBA tie together cooperating com- 
puter applications exchanging data that 
will probably never be read by anyone. 
Neither of these technologies will replace 
the other, but instead they will increas- 
ingly be used together. 
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Listing One 


<module> 
<name> banking </name> 
<interface> 
<name> savings_account </name> 
<inherits> bank_account </inherits> 
<operation> 
<name> deposit </name> 
<param> 
<name> amount </name> 
<type> money </type> 
<direction> in </direction> 
</param> 
</operation> 
</interface> 
</module> 


Listing Two 


module banking 
{ 


interface savings_account : bank_account 


void deposit (in money amount) ; 
i 
a3 


Listing Three 
//given Object bankObj 


JavaToXML translator=new JavaToXML(); 
String xml=translator.toXML(bank0bj) 


Listing Four 


String[][] accountInfo=new String[1] [1]; 
accountinfo [9] [0]="Red R Hood"; 
System.out.println(xml.toXML(accountInfo) ); 


Listing Five 


«java.lang.String dimension=2 length=1> 


100 


<item length=1> 


<item> Red R Hood </item> 


</item> 
</java.lang.String> 


Listing Six 


<servicename> : \ChicoSpuugBeest: ddjdem: : : IR: Bank</servicename> 


<method>accountList</method> 


<argument> 
<arg flag=in> 


<customerId>123456ABD</customerId> 


</arg> 
<arg flag=in> 


<accountNo> * </accountNo> 


</arg> 
</argument> 


<returntype>CustomerAccountSeq</returntype> 


Listing Seven 


interface accountView 


{ 


exception reject { 


string reason; // error text}; 
attribute string userId; 
attribute string password; 
attribute string customerId; 
attribute float accountNo; 


account accountLogin (in userId, inpassword) 
raises (reject, invalidLogon) ; 
account accountList (in customerId, in accountNo, 
out sequence CustomerAccountSequence) 
raises (reject, invalidCustomerID) ; 
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Magic 
Michael Swaine 


ny sufficiently advanced technology, 
Arthur C. Clarke said, looks like mag- 
ic. The rate of technological advance, 
the loose interpreters of Gordon 
Moore say, doubles every 18 months. Mul- 
tiply Clarke by Moore and the conclusion 
is inescapable— a day will come when 
current technology looks like magic. 

The premise of this month’s column is 
that that day is upon us. The magic mo- 
ment was reached in physics much earli- 
er, of course. It came early in the centu- 
ry when, trusting their math and their data 
more than their common sense, physicists 
accepted that the fundamental nature of 
the world is not only different than any- 
one thought but different than anyone can 
think. The quantum moment. 

Technology is humbler than quantum 
physics, though, and only occasionally 
does it take a leap so large that the mass 
of people can’t see the connection be- 
tween the new technology and older, ac- 
cepted (even if not really understood) 
technology. The telephone or telegraph 
may have been one such magic technol- 
ogy, but radio was not— it was original- 
ly known as “the wireless,” defined in 
terms of existing technology. It’s like some- 
thing you already know, but with waves 
in place of wires. Television was radio 
with pictures. Movies were moving pic- 
tures; that is, defined in terms of photog- 
raphy. New technologies may be startling 
when you first see them, amazing that 
whatever it is could even be done, but 
usually they are understood as extensions 
of known technologies. Some technolo- 
gies, like lasers, seem to come from 
nowhere, and seem a bit more magical. 

Now, though, the pace has picked up. 
Computers are the reason. Children who 
in a past era would have been taught to 
use particular tools are now being taught 
how to build tools. Steve Wozniak has 
been quoted as saying that computer sci- 
ence is the most important discipline that 
children can be taught. As a computer sci- 
entist, one is a litthe embarrassed by such 
grandiose claims, but is there any more 
powerful tool to place in anyone’s hands 
than a knowledge of programming? 

It’s a tool that’s being handed to a lot 
of kids. This has been happening, at an 
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increasing pace, for long enough for gen- 
erations of programmers to have made 
contributions to the advancement of tech- 
nology. No wonder the pace has picked 
up. No wonder it’s starting to look like 
magic. But that’s my premise, and I haven’t 
supported it yet. Maybe the following ob- 
servations will prove the point. 


Attaining Invisibility 

The goal of MIT’s Oxygen project (described 
in Scientific American, August 1999, and 
http://www.lcs.mit.edu/anniv/) is to push 
computer and information technology be- 
yond what passes for ease-of-use today, 
making it not only accessible to everyone, 
but richly exploitable by everyone. Michael 
Dertouzos, director of MIT’s Laboratory for 
Computer Science, envisions a world in 
which the technology we now access by 
typing and staring at screens in boxes on 
desks will be accessed via speech directed 
to the walls or to a tricorder-like device that 
everyone will carry. Thirty MIT faculty are 
involved in the five-year project to devel- 
op Oxygen, which Dertouzos hopes will 
become as pervasive as the air. in the 
speech in which he laid out the Oxygen 
project, Dertouzos predicted a 300-fold in- 
crease in human productivity in the next 
century. In the Scientific American article, 
this has been scaled back to a modest 300 
percent increase. I’m pretty sure Dertouzos 
meant 300-fold, or 30,000 percent.) 

Oxygen comprises eight technologies: a 
handheld device called Handy 21, an in- 
wall or in-car-trunk device called Enviro 
21, a network for collaboration called Net 
21, speech recognition, intelligent infor- 
mation retrieval, tools that allow anyone to 
script the devices and automate repetitive 
tasks, intelligent and informed tools for col- 
laboration, and the ability to customize the 
technology to your own needs. One of the 
features of the current technological land- 
scape that is not included in the vision is 
shrinkwrapped software. Apps will go 
away, Dertouzos predicts. 

Dertouzos’s eight technologies are tech- 
nologies that we would probably all agree 
are desirable, and that are being worked on 
in places other than MIT already. But what 
makes them special, and makes the Oxy- 
gen project interesting, is Dertouzos’s claim 
that these are the only important technolo- 
gies. Actually, it’s just the last five technolo- 
gies that he makes this claim about. He says: 
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The five technologies of speech (and oth- 
er perceptual capabilities), knowledge ac- 
cess, automation, collaboration and cus- 
tomization are the only new kids on the 
block. Out of the thousands of things that 
we can imagine doing in the new world of 
information, these five are the foundations 
on which any new activities that help us 
do more by doing less will be built. 


Doing more by doing less. It’s Dertouzos’s 
mantra. And when it works, it’s magic. 


Quicker Than the Eye 

One technology crucial to the Oxygen pro- 
ject is the Raw microchip, the project of 
Anant Agarwal of MIT. The idea is to get 
rid of the traditional interface to the mi- 
croprocessor— the instruction set— and 
let the software have access to the raw 
hardware. 

The Raw chip is an array of identical 
tiles, each containing memory and func- 
tion units, and each connected to its neigh- 
bors by wires. Excess logic gates at the 
junction between tile and wire make up 
a programmable switch. The (software) 
compiler can program those switches to 
direct the flow of signals through the chip. 
Among other things, the software can cre- 
ate registers as needed, storing data in a 
convenient spot to break up long wires 
that would incur long delays. 

Agarwal’s group has built a Raw simula- 
tor and written a compiler to take advan- 
tage of it. In a test application, the compil- 
er achieved a 10-fold speed increase over 
any existing conventional processor. Stu- 
dents tweaked the code for a further 10-fold 
increase. Agarwal hopes to see compilers 
developed that get that full 100-fold speedup 
when the chip becomes a real device. 


Doing the Impossible 


...trying to find a computer simulation of 
physics, seems to me to be an excellent 
program to follow out...and I’m not happy 
with all the analyses that go with just the 
classical theory, because NATURE ISN’T 
CLASSICAL, dammit, and if you want to 
make a simulation of nature, you’d better 
MAKE IT QUANTUM MECHANICAL, and 
by golly it’s a wonderful problem because 
it doesn’t look so easy. 

—Richard Feynman (1981) Unternational 

Journal of Theoretical Physics, v.21, p. 486) 


The open source movement invades all 
corners of computer science. Now, open 
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source has moved into quantum space 
with the open source quantum comput- 
ing project. The goal of the project is to 
simulate a quantum computer on a con- 
ventional one. The web site for the pro- 
ject Chttp:// www.openqubit.org/) has 
become a portal for quantum comput- 
ing work. 

Actual code has been released re- 
cently. There are now in fact several 
quantum computer simulators, and the 
field has become so crowded that it re- 
quires reviewers. A review of all exist- 
ing simulations at press time has been 
posted at http://www.dcs.ex.ac.uk/ 
~jwallace/simrevab.htm. Quantum com- 
puting— quick refresher here — promis- 
es to surpass the theoretical limits of con- 
ventional computing by taking advantage 





Give us 3 di 


of the quantum nature of reality. In a 
conventional computer, bits take on one 
of two discrete states. 

In a quantum computer, quantum 
bit— or qubits (pronounced “cue 
bits”)— exist in a quantum superposition 
of states when not being measured (mea- 
surement collapses each of them to some 
discrete state). 

This indeterminacy is the source of 
the computational power of quantum 
computers, which can be seen by con- 
sidering memory registers built of qubits. 
A quantum register of size 7 requires 2 
to the m complex numbers to represent 
its state, so it can store 2 to the 7 com- 
plex values, while a conventional regis- 
ter of the same size requires (and can 
store) just 7 integers. The storage ca- 
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pacity of quantum registers thus in- 
creases exponentially with size. That’s 
where the magic lies. 

Problems that can be solved in poly- 
nomial time are generally thought of as 
being tractable, while problems that can 
only be solved in exponential time are 
thought of as intractable. Quantum com- 
puting, by virtue of this exponential ex- 
pansion of storage capacity, promises to 
make the intractable tractable. 

In practice, this is exploited through 
quantum parallelism. The idea is to ap- 
ply a function to a quantum register, 
thereby exploiting the exponential ca- 
pacity of the register to get a lot of cal- 
culations simultaneously. Exactly how 
you exploit this capacity has been a bit 
of a puzzle. The problem is that you are 
essentially looking at the outputs corre- 
sponding to all possible combinations 
of inputs. The trick, then, is to make the 
output of all the possible inputs turn out 
to be something interesting, and useful. 
Peter Shor and L.K Grover, among oth- 
ers, have come up with algorithms that 
do just that. 


Is This 

Your Number? 

It was Shor’s algorithm for factoring large 
numbers, developed at Bell Labs in 1994, 
that brought about a significant shift in 
the perceived seriousness of the field of 
quantum computing. Shor’s algorithm, 
if implemented on a quantum comput- 
er, would crack many encryption 
schemes in polynomial time, making 
them worthless. 

This is what most of the quantum sim- 
ulation work has focused on— simulat- 
ing Shor’s quantum algorithm for fac- 
toring large numbers. 

It turns out that the hard part of fac- 
toring a large number can be reduced to 
finding the period of a particular peri- 
odic function involving the number. 
Shor’s algorithm finds the period of that 
function. 

The number to be factored is n; x is 
an integer coprime to n, the period of 
the function is represented by 7, and the 
function itself is x to the a mod n. 

The algorithm uses two quantum reg- 
isters. It loads register 1 with an equally 
weighted superposition of all integers from 
Q to g—1, where q is a power of 2 between 
n squared and 1 squared times 2. It calls 
the contents of register 7 a. Then it ap- 
plies the transformation x to the a mod n 
to register 1 and stores the result— a su- 
perposition of all outputs— in register 2. 
Due to quantum parallelism this takes only 
one step. It then measures the second reg- 
ister, observing some value k. The super- 
position of output values has collapsed 
into the value k. Quantum magic also has 
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Seagate Software NSMG and Telebackup Systems, Inc. 
have merged with VERITAS Software to create one 
VERI COOL Company (NASDAQ:VRTS)! 
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Fill out a Member Profile — we do 
the rest. This amazing opportunity 
is brought to you by Career 
Central for Developers, the world's 
easiest-to-use high-technology 
search solution. Please visit us at 
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and complete a Member 
Profile today. The 
sooner you do, the 
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you the job you 
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the effect of forcing the contents of reg- 
ister 1 into a state consistent with the val- 
ue measured in register 2. This can still 
be a superposition of values, but the val- 
ues all have to satisfy x to the a mod n 
equals k. 

The algorithm then computes the dis- 
crete Fourier transform on register 1. 
Again, this is a one-step operation, thanks 
to the magic of quantum parallelism. This 
has the effect of placing register 1 in a 
state where observing its value gives a 
number that has a high probability of be- 
ing a tractable function of 1, the desired 
period of the function. Anyway, that’s 
what they tell me. 

A conventional computer then uses 7 to 
finish the factorization. Please don’t build 
a quantum computer and try to run Shor’s 
algorithm on it based on my description; 
I undoubtedly got something wrong. I just 
wanted to give you the flavor of it, which 
is all I was able to extract from the pri- 
mary sources. 

Meanwhile, though, research into actu- 
ally building a quantum computer, not just 
a simulation, slogs along at various re- 
search sites throughout the world. 

Some labs have actually created work- 
ing qubits, which is no mean accom- 
plishment. Yasunobu Nakamura and his 
colleagues at NEC Fundamental Research 
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Labs in Tsukuba, Japan, have come the 
farthest to date in constructing a device 
capable of remaining in the necessary 
quantum state long enough to do some 
computation (all of two nanoseconds). 
Their device uses a superconducting quan- 
tum dot held at a temperature of 0.03K, 
so don’t look for the home hobbyist kit 
version any time soon. The next step in 
the gargantuan bottom-up process of 
building a quantum computer is to figure 
out how to wire up a few qubits to make 
a logic gate. And as the Raw microchip 
project shows, you can do a lot with log- 
ic gates. 


Sun’s Latest Trick 

Sun has released details of its MAJC (pro- 
nounced “magic,” which stands for “Mul- 
tiprocessor Architecture for Java Comput- 
ing”) chip architecture. As I understand it, 
MAJC was developed to move and ma- 
nipulate nontext data efficiently and to 
promote Java, although that’s not exactly 
how they put it in the presentations, which 
describe such MAJCal features as: 


e Multiple levels of parallelism. There’s 
parallelism in the handling of data 
through SIMD instructions, at the in- 
struction level via multiple functional 
units per processor, at the thread-of- 
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execution level via support for multi- 
threading, and at the system level be- 
cause you can have multiple MP pro- 
cessors on a single chip. 

Software portability. MAJC chips are de- 
signed for installation-time or JIT com- 
pilation of code. Compatibility across 
generations is regarded as somebody 
else’s responsibility; it won’t be a con- 
straint on the design of future chips. 
Modular architecture. Because Sun sees 
MAJC chips being used in everything 
from screen phones to medical imaging 
devices, the technology employs a mod- 
ular architecture: One implementation 
might be just one processing unit and 
two functional units with unneeded in- 
structions turned off, while another 
might involve many processing units, 
each with four functional units. The de- 
sired design would be put together from 
modular components. 

Other stuff. It’s based on a Very Long In- 
struction Word approach. More news 
about it will have been released by the 
time you read this, but this much seems 
clear: while this may be a very useful de- 
vice, it will not in and of itself seem very 
magical. The applications of it might. 
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t's August in the mountains of central 

eastern Pennsylvania. We’re here in the 

RV for Judy’s annual family reunion, to 

be hosted this year at the farm where 
she grew up. The drought ended last week 
with some heavy rain, and everything looks 
green again, although the crops that my 
father-in-law planted this spring suffered 
the effects of too much heat and too little 
water. There’s not much to pick. 

The farm sits on the southern side of a 
small mountain range overlooking a wide, 
green valley. The tiny town of Bran- 
donville nestles at the base of the valley. 
During the winter when the trees are bare, 
you can see a few rooftops down there. 
This time of year only a distant neighbor’s 
farmhouse reminds you that there is any- 
one else in the world at all. If you are ac- 
customed to the sounds of a city or live 
anywhere near an interstate highway you 
would be stunned by the silence here. My 
father-in-law is 80, has been a farmer all 
his life, and has owned and worked these 
40 acres for the better part of 50 years. 
His comment when I marvel over the spec- 
tacular view is, “If it wouldn’t be for those 
damn trees, maybe we could see some 
more of the town.” 

This trip brings to a conclusion a two- 
week visit to our house in Florida by our 
four grandchildren, ages five, seven, sev- 
en, and nine. They returned with us to re- 
join their parents who look rested and 
content. You don’t want to know how I 
look. I'll spare you the details of what it’s 
like to keep four little people entertained, 
fed, clean, well-behaved, out of trouble, 
and from killing one another from morn- 
ing to night when you are not used to do- 
ing it and they know it. I will say this 
much: If you want to know anything at 
all about the Cartoon Network, Power 
Rangers, or Pokémon, just ask me. 


Al is a DDJ contributing editor. He can be 
contacted at astevens@ddj.com. 


bttp://www.ddj.com 


That hectic visit was not the only har- 
ried experience brought to a close this 
week. I just completed work on the sixth 
edition of a C++ tutorial book called Teach 
Yourself C++, published by MIS:Press, which 
is now owned by IDG Books, who pub- 
lish all those For Dummies... books. (There 
is a C++ for Dummies, written by someone 
else, but I’ve never read it. I don’t believe 
that dummies can do C++.) The first five 
editions of my book reflect the evolution 
of the C++ programming language as it has 
been implemented on PC platforms and as 
the ANSI/ISO standardization effort has pro- 
ceeded. I didn’t plan it that way. The lan- 
guage kept growing and changing, the 
compilers kept upgrading, and I kept re- 
vising the book. The sixth edition will be 
the first one to be published since the Stan- 
dard was approved. 

The sixth edition departs from the tra- 
ditional book-with-code format that I used 
in the first five editions. Some publisher- 
type suits in positions of influence took 
some of the money they made by selling 
books to all those dummies, acquired 
MIS:Press, took a look at its inventory of 
titles, and decided that what had been a 
best-seller needed to be fixed and made 
more “visually appealing.” Publishers, you 
see, know much better than authors and 
readers what constitutes a good pro- 
gramming book and what appeals to pro- 
grammers. I don’t know how they know 
that, but they do. Just ask one of them. 

The new layout uses what they call a 
“spread” format. A spread comprises two 
facing pages with narrative text and snap- 
py “visual elements” like hints, references, 
and notes on the left page and more vi- 
sual stuff— figures, screen shots, and so 
on— on the right. Each such spread teach- 
es what they call a “task” or “concept.” 
They developed this format for books that 
teach things that have many visual ele- 
ments, such as applications, HTML, and 
Visual Basic, and they think that it should 
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work for advanced programming language 
books, too. Programming books for the 
MTV generation, the sound bite set, the 
People magazine crowd. Short sentences, 
lots of eye candy. Too many words and 
not enough pictures scare those dummy 
book buyers off. Before long, computer 
books will include a companion package 
of crayons instead of a CD-ROM. 

Visual appeal, that’s what sells. But with 
a generic C++ book, the only thing visu- 
al is source code and program output. Dis- 
creet lessons about C++ often involve 
more code than you can get on a page, 
or more text, or too little code to have its 
own page, or too little text, or some com- 
bination of all that. Rarely does a specif- 
ic lesson involve exactly enough text and 
code to fit this new, visually appealing, 
Johnny Bravo, dumbed-down format. Try, 
for example, to fill two pages explaining 
and demonstrating what an int is and how 
to instantiate one. Try to explain and 
demonstrate “explicit specialization of a 
member of a class template partial spe- 
cialization” in only one page of text and 
30 lines of code. Try explaining to a de- 
velopment editor that there are no pretty 
CROSS REFERENCE visual elements with 
this concept because THERE IS NOTH- 
ING TO CROSS REFERENCE. Duh. Con- 
sequently, an author spends a great deal 
of time contriving content in some cases 
and sacrificing content in others to force 
the problem to fit the solution. Writing this 
book was not an easy experience. I'll take 
rowdy grandchildren every time. 

Don’t feel too sorry for me, though. I 
whined a lot throughout the project, and 
they gave me more latitude than I think 
they would have liked. Mine was the first 
programming book in the new format and 
we were learning about it together. 

It was my hope that the sixth edition of 
Teach Yourself C++ would be the last one, 
particularly given the pain that writing it in- 
volved. The C++ Standard is approved and 
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the language is well-defined and under- 
stood. However, the last several editions of 
the book included compilers on the com- 
panion diskette and, later, the companion 
CD-ROM, and I am not willing to sacrifice 
that feature. As we speak, I do not have a 
standard conforming C++ compiler and li- 
brary that I can freely distribute. The fourth 
edition included the DOS DJGPP port of 
the GNU command-line compiler. The fifth 
edition included the Cygnus gnu-win32 port 
of the GNU C++ compiler along with a 
front-end Windows 95-hosted IDE named 
Quincy 97 that I developed for this pur- 
pose. The sixth edition will include an im- 
proved Quincy 99 with the mingw32 port 
of the Egcs 1.1.2 compiler, which is a 
closely conforming C++ compiler. Its li- 
brary, however, is still the old one, so my 
book, which purports to be about Stan- 
dard C++, is unable to demonstrate some 
nifty features of the Standard C++ Library, 
such as stringstreams, wide-character 
streams, and so on. This means that when- 
ever they get all that stuff working, I'll be 
looking at doing a seventh edition. Oh, 
the pain. 


Frameworks, Graphics, and C++ 
Although my new edition promises to 
be a hot seller (according to the suits), 
most of you will not need to read it. Most 
readers of this column already know 
C++. Many of you have told me that you 
learned C++ from earlier editions of the 
book. (You know, the ones in the old 
format that wiser heads assure me no 
one would ever buy.) But during its de- 
velopment, I learned some new and bet- 
ter ways to teach some difficult subjects. 
Difficult because arcane concepts need 
concrete examples. 

One of the things that the mingw32 port 
of Egcs adds to the compiler is support 
for Windows applications development 
with the Win32 API. Ho hum. But wait. 
This feature opened a window of oppor- 
tunity that I did not have with earlier edi- 
tions— the ability to demonstrate many 
of the features of C++ and object-oriented 
programming by using simple graphical 
applications. 

In the old days, if you wanted to write 
about graphics on a PC platform, you 
had to spend some time talking about 





Figure 1: Using the TYFC framework. 
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graphics programming book from that 
time and you'll find lots of details that 
you don’t want to know about. The authors 
had no choice. If they were going to get 
their example programs running and dis- 
playing graphical images, they had to deal 
with the hardware. GUI platforms obviate 
all that low-level stuff by putting a device- 
independent layer between the program- 
mer and the hardware. 

A C++ tutorial book should teach C++, 
not some GUI API. But there is a com- 
pelling reason to use a GUI to teach 
C++. Nothing demonstrates the proper- 
ties of object-oriented programming tools 
better than a simple graphical class li- 
brary and a GUI simplifies the graphical 
parts. Every programmer who has ever 
written or used graphical rendering pro- 
grams that draw shapes on the screen 
can immediately understand polymor- 
phism when you explain it in terms of 
abstract and concrete shape classes. The 
abstract base shape class knows noth- 
ing about specific graphical shapes oth- 
er than the properties that they all 
share — position, rotation, and so on. 
Concrete shape classes derived from the 
abstract shape know how to draw, scale, 
transform, and rotate themselves. Early 
works on OOP missed the boat on this 
one by trying to use examples about fur- 
niture, insects, toasters, and such to ex- 
plain class hierarchies. These are not 
things that you typically deal with. 
Graphical objects are, and they are vi- 
sual things that are easily demonstrated 
with concrete examples. More “visually 
appealing,” don’t you know. 

There is another reason to use a GUI 
to teach C++. You don’t want to burden 
the student with details of the API, which 
is a course in itself, yet the example pro- 
grams need the API to display the win- 
dows. To hide those details, you can pro- 
vide a simple framework. Most C++ 
programmers use GUI frameworks such 
as MFC. By offering early exposure to a 
simple framework, you are preparing the 
student for the real world. 


TYFC 

To teach all these things, I built a simple 
framework called TYFC, for “Teach Your- 
self Framework Classes.” Catchy, huh? The 
version I discuss here is more advanced 
than the one in the book. I have experi- 
mented with TYFC to further examine its 
potential as a teaching aid, and this dis- 
cussion reflects that research. 

TYFC implements an abstract base class 
named TYFCApp, from which the appli- 
cation program derives a class. TYFCApp 
is small, consisting of one header file, 
tyfc.h (available electronically; see “Re- 
source Center,” page 5), which declares 
the TYFCApp class, and one source code 
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file, tyfe.cpp (available electronically), 
which defines the member functions. You 
do not need to understand the imple- 
mentation details of TYFCApp, which 
teaches the lesson that hiding details of 
implementation raises the programmer’s 
level of abstraction. 

You are allowed to peek, however. An- 
other lesson to learn about C++ is that al- 
though the details are said to be hidden, 
they are actually mostly in view in the 
class declaration and completely in view 
in templates. You can hide nontemplate 
member function definitions, but I prefer 
that the source code be available, if only 
for documentation. 

The application program derives a class 
from TYFCApp. The derived class repre- 
sents the application, and its constructor 
specifies to the base class constructor the 
size of the application window and its cap- 
tion. The derived class overrides the base 
class’s pure virtual Initialize function. The 
program instantiates an external object of 
the derived class. As a result of that in- 
stantiation, TYFCApp creates the window 
and calls the derived class’s Initialize func- 
tion, which must put together something 
to display. textgl.cpp (available electron- 
ically) demonstrates the application pro- 
gram’s use of the framework. Observe that 
the application has no main function. The 
framework provides it, in this case, a Win- 
Main function, a detail that TYFC hides 
from you, another lesson to be learned. 
The WinMain function ensures that there 
is an instantiated derived TYFCApp object 
somewhere (its constructor records that 
fact, which is why the application object 
must be external), creates a window, and 
enters the usual message dispatching loop. 
The application program’s Initialize func- 
tion instantiates some graphical objects 
and calls addshape to add them to the dis- 
play, in Figure 1. 

TYFCApp, being a teaching vehicle, is 
only for static graphical displays, although 
it could evolve into a much larger frame- 
work. It knows about an abstract base 
class named Shape, and it maintains an 
std::list<> container of pointers to Shape 
objects, teaching the student that standard 
containers are easier to use than home- 
grown ones. TYFCApp also demonstrates 
polymorphism. Whenever the run-time 
system sends the WM_PAINT message to 
the application window, TYFCApp calls 
the draw function for each of the Shape 
objects pointed to by entries in the list 
container. The draw function in the Shape 
class is a pure virtual function, so the 
pointers have to be pointing at objects of 
classes derived from Shape, and those 
classes must have overriding versions of 
the draw function. 

I used an std:.list<> container for the 
Shape pointers, because a future version 
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(continued from page 108) 

of the framework might need to know 
about and maintain the Z order of graph- 
ical objects. A list container is a good ve- 
hicle for reordering the objects it contains. 
Another lesson taught. 

All TYFCApp knows about graphics is 
how to list Shape pointers and call their 
draw functions and how to draw a sin- 
gle point at an x/y coordinate on the 
screen. It adds shapes to its list when the 
derived class’s Initialize function calls 
the addshape function. The application 
must use a graphical class library to ex- 
ploit these characteristics. Win32 supports 
higher graphical abstraction, but TYF- 
CApp’s purpose is to teach at lower lev- 
els. All we want the graphical engine to 
do is open a window let us draw pixels. 


eraphics.h derives classes from Shape, 
and it has several lessons to teach. First is 
the example of concrete classes defining 
the polymorphic behavior of their com- 
mon abstract base class, which is the orig- 
inal reason for these libraries. But other 
lessons fell out of the exercise. 

The first such lesson is about a dif- 
ferent kind of abstraction, the kind you 
implement with templates. Templates 
aren’t just for containers, they are also 
good at expressing abstractions. In this 
case, the graphical library in graphics.h 
does not need to associate with the spe- 
cific graphical engine that TYFCApp im- 
plements. That detail is provided as a 
parameterized type, which makes graph- 
ics.h and its concrete shape classes 
portable to other graphical engines. The 


Software developers: 














Youd 


















Chart FX 98 has 7 ways 
to help you balance 


your baby. 


COM makes app develop- 
ment easier than ever! 


ya Easy to understand API 
makes programming 
child’s play! 


Wizards help you create 
charts right out of the box! 


New user interface makes 
application development 
even easier! 


Change chart properties 
and attributes with the click 
of a mouse! 


New color palettes 
end-users will love! 


14 More chart types 
than ever before! 


Chart FX 98— 

The powerful 32-bit charting 
component you and your 
baby have been waiting for! 


v.softwarefx.com 


000.392.4278 


Chart FX 98 is available only in a 32-bit version. Chart FX 3.0 is still available for 16-bit applications. Chart FX 3.0 


and Chart FX 98 are royalty-free for distribution within your applications. For more information, call 800.392.4278. 
©1998-1999 Software FX. All rights reserved. Visual Basic and COM technology are trademarks of Microsoft Corp. 


110 


Dr. Dobb’s Journal, November 1999 





engine and the library share only the 
knowledge of the abstract base Shape 
class (available electronically), which is 
independent of either of them. To port 
graphics.h to a different graphical en-— 
gine, you could put a wrapper around 
the engine to contain and manage shape 
pointers, provide the addshape and 
drawpoint functions, call the applica- 
tion’s Initialize function, and call the 
shape objects’ draw functions. 

The next lesson is about specializa- 
tion. Given that some graphical engines 
support higher level drawing functions, 
you can specialize the draw member 
functions for instantiations with those 
engines. Graphics.h implements the Bre- 
senham line-drawing algorithm and the 
usual circle-drawing algorithm that plots 
points around the circle by using radians, 
sines, and cosines. These algorithms are 
nice for teaching how graphical objects 
are drawn, but they are slow and usually 
unnecessary. You can specialize the draw 
member functions for the Line and Circle 
template classes when they are instantiat- 
ed for graphical engines that draw these 
things for you. 

The last lesson from graphics.h is about 
namespaces. Graphics.h defines a Line 
class, a Rectangle class, and a Circle class. 
These are nice, meaningful names— the 
kind of names you like to give to things 
in a program. There is a catch, however. 
Somewhere in the mingw32-egcs 1.1.2 
implementation of the Win32 API global 
namespace is something named Rectan- 
gle. Graphics.h puts its classes in a 
namespace to isolate its names from the 
global namespace and, consequently, 
demonstrates the utility of the C++ 
namespace feature. 

The further I get into this specific 
teaching paradigm, the more I learn 
about how a simple graphics library can 
teach C++. Eventually the library will be- 
come too complex to serve that purpose, 
whereupon it might become a mecha- 
nism for teaching graphics programming 
in C++. And thus what started out to be 
a simple example of a complex subject 
surprises us when it demonstrates 
through its own growth how we learn 
things and how we ought to teach things. 
Sometimes you just can’t see the town 
for the trees. 
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Does Java Support 
Design by Contract? 


Krishnan Rangaragjan 


esign by Contract (DBC), pioneered 

by Bertrand Meyer (see Object- 

Oriented Software Construction, Sec- 

ond Edition, Prentice Hall, 1997), is 
widely acknowledged to be a powerful tech- 
nique for writing reliable software. The three 
key elements of DBC are preconditions, 
postconditions, and class invariants. An ex- 
ample of a language that has direct support 
for DBC is Eiffel. Unfortunately, Java does 
not directly support DBC. However, there 
are third-party tools for Java that extend sup- 
port to Design by Contract. One such tool 
is Reto Kramer’s freely available iContract 
(http://www.reliable-systems.com/tools/). 
iContract parses Javasource code that is an- 
notated with assertions inside Javadoc-style 
comments and generates modified Java 
source code where the assertion checking 
logic is appropriately embedded. The ben- 
efits of this approach are: 


e Because assertions are embedded inside 
comments, they are transparent to the Java 
compiler. 

¢ No costly source modifications are need- 
ed to regenerate assertion-disabled byte 
code; simply recompiling the original 
source with a normal Java compiler is all 
that is needed. 


Two other equally interesting approach- 
es for incorporating DBC in Java are Hand- 
shake (see “Adding Contracts to Java with 
Handshake,” by Andrew Duncan and Urs 
Holzle, Technical Report TRCS98-32, De- 
partment of Computer Science, University 
of California, Santa Barbara, http://www.cs 
Jucsb.edu/TRs/ TRCS98-32.html) and jCon- 
tractor (see “jContractor: A Reflective Java 
Library to Support Design by Contract,” by 
Murat Karaorman, Urs Holzle, and John 
Bruno, Technical Report TRCS98-31, De- 
partment of Computer Science, University 
of California, Santa Barbara, http://www 
.cs.ucsb.edu/TRs/TRCS98-31.html). Hand- 
shake uses a separate contract file to dy- 


Krishnan is president of Man Machine 
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namically synthesize a class file that incor- 
porates assertions, whereas jContractor uses 
a design pattern approach. Each approach 
has strengths and limitations, and it is not 
my intention to discuss them. In this article, 
I'll present yet another approach, one that 
is part of a class-testing environment avail- 
able from Man Machine Systems. 

In a nutshell, our approach is this: We 
parse a Java source file suitably annotated 
with preconditions, postconditions, and in- 
variants inside Javadoc comments and gen- 
erate triggers in JMScript (a proprietary script- 
ing language that is built on top of Java). 
These triggers are then executed by run- 
ning the Java program (along with the trig- 
gers) by attaching the script interpreter to 
JVM via an extension DLL. 

The first step is to add preconditions, 
postconditions, and invariants to Java code 
within Javadoc comments. Special tags are 
used for this; Table 1 lists these tags. 


e Class Invariant (@inv) should be defined 
inside a Javadoc comment before the class 
definition. The marker is followed by a 
Boolean expression that may reference 
any element of the class or its direct/in- 
direct bases, including private elements. 
Precondition (@pre) must be defined with- 
in Javadoc comments preceding the re- 
spective methods. The marker is followed 
by a Boolean expression, as in the class 
invariant. Also, the condition may refer- 
ence arguments passed to the method. 

Postcondition (@post) is specified within 
Javadoc comments preceding the re- 
spective methods. The condition may be 
any Boolean expression, as in the pre- 
condition case. The Boolean expression 
may also use the qualifier Sprev on an 
expression to denote the expression’s val- 
ue at the method’s entry point. For ex- 
ample, $prev (top) denotes the value of 
variable fop at method entry (without the 
qualifier, it indicates its current value). The 
keyword $ret may appear in the condi- 
tion to denote the method’s return value. 


The assertions must conform to the syn- 
tax of JMScript Boolean expressions. To cap- 
ture more complex conditions, you can use 
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the special escape marker “@macro” to em- 
bed a JMScript code fragment. Table 2 shows 
how preconditions, postconditions, and in- 
variants, are used in the context of a class. 

We allow the contract to be specified for 
an interface, which gets propagated to its 
implementing classes. The same rule is ap- 
plied to class hierarchies. For both interfaces 
and class hierarchies, preconditions are dis- 
juncted, whereas postconditions and class 
invariants are conjuncted. 

Consider Listing One, the canonical stack 
implementation in Java (to avoid confusion 
with JDK Stack class, I’ve named the ex- 
ample MyStack). Once a Java source file has 
been annotated as the one just mentioned, 
the next step is to use the preprocessor jms- 
assert utility to generate JMScript triggers for 
the assertions. In this case, we run_jmsassert 
-s MyStack java. The preprocessor creates 
the output files default_MyStack.jms and 
Startup.jms. The first file (Listing Two) con- 
tains JMScript triggers corresponding to the 
embedded assertions in Java source. The 
second file (Listing Three) makes it conve- 
nient to register the automatically generat- 
ed triggers with JMScript at run time. 

The third step is to compile the relevant 
Java source files. Assume that in addition to 
the MyStack definition as just mentioned, 
we also have the test driver class in Listing 
Four. This driver, along with the MyStack 
class, is compiled using a regular Java com- 
piler such as “javac.” The final step is, of 
course, to run the Java code with assertions 
enabled. To do this, invoke the Java inter- 
preter using: java-Xdebug -Xnoagent 
-Djava.compiler= NONE -Xrunjmsdll: Start- 
up StackTest. The CLASSPATH environment 
variable must be appropriately set to JDK1.2 
run-time files, and additionally, must include 
mmsclasses.jar, which is supplied as part of 
the assertion environment. 

The DLL jmsdll includes the JMScript in- 
terpreter. This DLL registers itself with JVM 
and assigns assertion triggers to the re- 
spective Java methods. When the JVM in- 
vokes a method, the call is intercepted by 
the DLL, and if a trigger (precondition, post- 
condition, or invariant) is associated with it, 
the corresponding JMScript trigger method 
is invoked. The trigger code in JMScript can 
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access private elements of a class. This is a 
significant advantage in terms of testing. 
Though JMScript is primarily useful for bring- 
ing DBC to Java, it’s designed for general- 
purpose scripting, and has some attractive 
features such as partial function specializa- 
tion, multimethod, dynamic inheritance, and 
so on. Its strength derives from the under- 
lying JVM. (For more details on JMScript, 
go to http://www.mmsindia.com/.) 

One of the benefits of this approach is 
that the original Java source code is un- 
modified; what is tested with assertions is 
the same as what is executed normally. To 
run the test driver without enabling asser- 
tion code, simply run: java StackTest. The 
limitation that Java source must be available 
in order to specify contracts is overcome 
when you use JVerify (a tool that supports 
invasive testing by executing test drivers 
written in JMScript). That environment al- 
lows assignment of triggers to Java by in- 
specting compiled class files. 


Conclusion 

Although Java does not yet directly support 
Design by Contract, there are several ways 
to add such support. The approach I've pre- 
sented uses a preprocessor to map contracts 
embedded in Java source code to triggers 
in the scripting language. Triggers are then 
automatically executed by an extension DLL 
that includes the JMScript interpreter. To run 
without assertions, all that is required is to 
run the Java program without the extension 
DLL. This approach ensures that the released 
program is identical to the one tested. To 
test a class, no modifications are necessary 
(other than source annotation). 
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Listing One 
@inv (top >= ® && top < max) 


* 
class MyStack { 
private Object[] elems; 
private int top, max; 
/** 
@pre (sz > B) 
@post (max == sz && elems != null) 


* 
public MyStack(int sz) { 

max = SZ; 

elems = new Object[sz]; 


| *% 
@pre !isFull() 
@post (top == $prev (top) + 1) && elems[top-1] == obj 


*/ 

public void push(Object obj) {f 
cleasitoptt] = obj; 

/** 


@pre !isEmpty() 
/ @post (top == Sprev (top) - 1) && $ret == elems[top] 
* 


public Object pop() { 

: return elems[--top] ; 
| ** 
- @post ($ret == (top == max) ) 


public boolean isFull() { 
return top == max; 


ie 
iv @post ($ret == (top == O)) 


public boolean isEmpty() { 
return top == 0; 


j 
} // End MyStack 


Listing Two 

/* Trigger file for class #default.MyStack. Generated by JMSAssert on 
* Monday, April 12, 1999. Any changes you make to this file will be 
pee if you regenerate this file. 
* 

import macro; 


// Postcondition for method - MyStack(int) 
MyStackPost(meth, $obj, sz) { 
assertPost ((Sobj.max == sz && Sobj.elems != null)); 


// Precondition for method - MyStack(int) 
MyStackPre(meth, Sobj sz) { 
assertPre((sz > B)); 


// Postcondition for method - void push(Object) 
pushPost (meth, Sobj, obj, $ret) { 
assertPost((S$obj.top == 





// Precondition for method - void push(Object) 
pushPre(meth, Sobj, obj) { 
this.top$prev = $o i; £0R% 
assertPre(!Sobj.isFull()); 


// Postcondition for method - Object pop() 
popPost(meth, Sobj, $ret) { 
assertPost((S$obj.top == this.top$prev - 1) && $ret ==_ 
Sobj .elems [$obj.top] ); 


} 
// Precondition for method - Object pop() 
popPre(meth, $obj) { 

this.top$prev = $obj.top; 

naactttret ticki. 1taev): 


// Postcondition for method - boolean isFul1() 
isFullPost(meth, $obj, $ret) f ; 
assertPost((Sret == (Sobj.top == $obj.max))); 


// Postcondition for method - boolean isEmpty () 
isEmptyPost(meth, Sobj, $ret) f{ 
assertPost((Sret == (Sobj.top == 9))); 


MyStackinv(meth, $Sobj) { 
assertInv((Sobj.top >= 8 && Sobj.top <= $obj.max)); 


static { 
assertStrMyStack = { 

{ “<init>(1I)V", “POSTCONDITION", “MyStackPost" }, 
{ "<init>(I)V", “PRECONDITION”, “MyStackPre" }, 
{ seg patel po fie aed ta "POSTCONDITION", “pushPost" }, 
{ “push(Ljava/lang/Object;)V", “PRECONDITION”, “pushPre" }, 
{ “pop()Ljava/lang/Object;:", “POSTCONDITION", “popPost" }, 
{ “pop()L ava/lang/Object;", "PRECONDITION", “popPre" }, 
{ “isFull()Z", “POSTCONDITION", “isFullPost" }, 

{ gg oo AL “POSTCONDITION", “isEmptyPost™ }, 

{ "", “INVARIANT", “MyStackinv" } 


33 
; setClassTrigger ("MyStack", assertStrMyStack) ; 
load() {} 


Listing Three 
//Startup trigger file - c:\ranga\jverify\Startup. jms 
import macro; 
load() {} 
static { 
default_MyStack.load(); 


Listing Four 
class StackTest { 
public static void main(String [] ares) { 
MyStack s = new MyStack (2); // push at most two elements 
s.push(new Integer (1) ); 
s.push(new Integer (23)); 
s.push(new Integer(@)); // Precondition violation here! 
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Adaptive 


Unsharp Masking 


Tim Kientzle 


nsharp masking is a photographic 

technique that increases the sharp- 

ness of a photographic image. It’s 

heavily used by astronomers for im- 
proving blurry telescope images. More re- 
cently, it’s become a standard tool among 
graphic artists for enhancing blurry 
scanned images. 

In the darkroom, you create what is 
called an “unsharp mask” from a large- 
format negative by exposing another piece 
of film using a spacer. The result is an un- 
derexposed, blurry positive. When you 
sandwich this with the original negative, 
some of the fuzziness is canceled out, re- 
sulting in a sharper image. 

The digital version of this process is ac- 
tually much simpler in many ways. To 
demonstrate the technique, pretend for a 
moment that you’re an astronomer trying 
to photograph a distant star. Figure 1 is a 
cross-section of the image you would like 
to get— a sharp-edged light disc against 
a dark background. But there are many 
kinds of interference to contend with, so 
your actual result is more like Figure 2. 

Just as in the photographic technique, 
the key is to take the actual image and 
subtract a blurred version of it from itself. 
Figure 3 shows the result in this particu- 
lar case. This unsharp mask is very faint, in 
this particular case, the maximum value is 
only about a hundredth of the peak mag- 
nitude of the original. This is an important 
observation; for example, the “Unsharp 
Mask” script included with version 1 of 


Tim is DDJ’s executive editor and can be 
contacted at kientzle@ddj.com. 


hitp://www.ddj.com 


GIMP (http://www.gimp.org/) builds an un- 
sharp mask using 8-bit graphics operations, 
which severely limits the enhancement you 
can obtain. 

The enhanced image is created by 
adding a multiple of the unsharp mask 
back to the original. In the case of Figure 
4, I've multiplied the unsharp mask by 100 
before adding it back to the original. The 
enhanced version has a much narrower 
peak than the original data. The sides of 
this peak are much steeper, which corre- 
sponds to a sharper image. 

You should note a few interesting facts 
about Figure 4. First, I created the unsharp 
mask using a blur radius much smaller 
than the blurring in the original; you can 
play with the parameters, but I think you'll 
agree that a relatively small radius gives 
the best improvement. Also, notice how 
the enhanced data here goes out of range 
(the red line goes below the x-axis). This 
is a problem with too-aggressive unsharp 
masking. You lose all detail in areas where 
this happens. One way to address this 
problem is to reduce the weighting when 
you combine the unsharp mask with the 
actual data. Of course, real images don’t 
have a single feature you're trying to en- 
hance, so it may be very difficult to find 
a single weight that simultaneously pro- 
vides the needed enhancement and does 
not destroy faint details. 


Adaptive Unsharp Masking 

The key to creating an adaptive unsharp- 
masking algorithm is that all of the pa- 
rameters apply to single pixels. It’s there- 
fore quite reasonable to vary these 


Dr. Dobb’s Journal, November 1999 









parameters from pixel to pixel to provide 
different levels of enhancement. To do 
this automatically, you first identify the 
problem areas. Figure 5 shows only the 
parts of the enhanced image that went 
out of range. 

To build Figure 4, I added 100 times the 
unsharp mask at every point. In the adap- 
tive version, I start with an array that con- 
tains 100 for every pixel. I then wanted to 
reduce the weight around problem areas. 
This is not an entirely trivial operation, I 
wanted to reduce the weight more in ar- 
eas that were more troublesome. I also 
wanted the weighting to vary smoothly 
across the image so that my adaptive 
weighting wouldn’t alter any details. (Af 
ter all, the point of having an adaptive al- 
gorithm is to preserve details.) One simple 
solution was to first add a multiple of Fig- 
ure 5, then apply a Gaussian Blur to the 
resulting array. Figure 5 shows more neg- 
ative in areas that are more of a problem. 
By blurring the weighting array, I ensured 


Figure 1: The data we wanted; a 
light-gray block against a dark-gray 
background. 
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Figure 2: A blurry version of Figure Figure 6: I initialized my array of 





1; note that the peak is much lower. Figure 4: Adding the unsharp mask weights to 100, subtracted a multiple 
7 7 to the test data yields an enhanced of Figure 5, and then applied a 
version (red), which I’ve shown Gaussian Blur. 
together with the actual data (blue) 
and the desired data (black). that the weighting would vary smoothly 


across the image. The resulting array of 
weights is shown in Figure 6. 

Once I had a suitable weighting array, 
I could multiply each pixel of the unsharp 
mask by a different weight and add the 
result to the original data. Figure 7 is the 
result. This final enhanced version does 
not go out of range. 

The basic approach here was simply to 
replace a single fixed parameter (the weight- 
ing used to combine the unsharp mask with 
the original image) with a separate param- 


Figure 3: The unsharp mask is the 
difference between the test data and a 
blurred version of that data. In this 
case, I used a Gaussian Blur with a 
radius of two, and subtracted the : a 
blurred version of Figure 2 from the Figure 5: The problem areas from eter for every pixel. Of course, this required 
original. Figure 4. juggling other parameters; I used a single 
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Figure 7: The final enhanced data 
(purple). This was created by 
multiplying the weighting array 
(Figure 6) by the unsharp mask 
(Figure 3) and adding to the original 
data (blue). This enhanced version 
does not go out of range. 


multiplier and a blur radius to construct the 
weighting array. To choose that multiplier 
automatically, you can simply increase it 
just to the point that none of your original 
image is out of range. I don’t know of a 
good way to automatically determine the 
blur radius for this operation. 

The basic unsharp-mask algorithm does 
have one other fixed parameter that could 
become adaptive. A Gaussian Blur is a 
pointwise operation, each pixel is replaced 
with a weighted average of neighboring 
pixels, with the weighting determined by 
a Gaussian distribution with a selected ra- 
dius. There is no technical reason the 
Gaussian radius cannot be varied at each 
pixel. To do so, you would need an ar- 
ray of Gaussian Blur radiuses that you 
could manipulate. The final blurred ver- 
sion would require, at every pixel, com- 
puting fresh Gaussian weights and ap- 
plying those weights at that pixel. This is, 
unfortunately, extremely computation in- 
tensive, and probably not practical for 
most purposes. I’ve provided a Mathe- 
matica notebook (available electronically; 
see “Resource Center,” page 5) that lets 
you experiment with the different param- 
eters. The notebook uses a Fourier trans- 
form to compute the Gaussian Blur, which 
is a fast method that unfortunately does 
not lend itself well to an adaptive ap- 
proach. (You can view the notebook us- 
ing MathReader 3.0, which is available at 
no charge from Wolfram Research; see 
http://www.wolfram.com/.) 


Conclusion 

The unsharp-mask weighting layer doesn’t 
necessarily have to be constructed auto- 
matically. An alternative approach would 
be to initialize it to all zeros, thus allow- 
ing users to paint on it. If carefully done, 
this would allow users to selectively sharp- 
en the image only where needed. 
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DR. ECCO’S OMNIHEURIST CORNER 


Calabaza 


Dennis E. Shasha 


e have found a remarkable tribe 
in the jungles of Venezuela,” the 
strongly built ethnobiologist Pan- 
ta de Leon began in slightly ac- 
cented English. De Leon had heard of 
Ecco from Natasha, the archaeologist of 
the Sudan. De Leon’s motivation, we all 
decided later, was much more pure. 

“The tribe has lived in near permanent 
isolation except for occasional, mostly un- 
friendly, exchanges with outsiders. They 
are fine singers and percussion musicians. 
Their principal musical instrument is a fruit 
gourd called (in Spanish) a Calabaza. This 
gourd is so central to their life in fact that 
they call themselves the Calabazas when 
meeting (friendly) outsiders. Recently, I 
discovered that they also use the Calabaza 
as a timepiece for the ritual they call— 
I’m translating roughly—'Prayers to the 
Clock God’. 

“Let me explain. Their’s is one of the 
wettest, most mosquito-infested corners 
of the world. This has kept them safe. It 
also means that anything made of metal 
either rusts or is quickly impregnated with 
water. 

“Many years ago, they found a group 
of explorers who had lost their way and 
then died of a poisonous fruit. One of the 
explorers had a watch, which the Cal- 
abazas brought back to the village. The 
watch worked for one moon, they say. 

“The first day they had it, the tribal el- 
ders tried to figure out the meaning of the 
timepiece and they quickly concluded that 
it must be used for prayer. The explorers 
died because they hadn’t prayed at the 
right time, they decided. Fascinated by the 
circular rhythm of the second hand, the 
elders directed the village artisan to cre- 
ate several gourds of the same size, each 
with a hole. They told the artisan that he 
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should create the gourds so that a full 
gourd would drain completely in one 
minute when the hole was opened. Un- 
fortunately, the village artisan found him- 
self unable to carve a large enough hole 
without destroying the Calabaza. Instead, 
he created Calabazas having two holes. 

“Opening one hole in a full Calabaza 
would cause the water to drain completely 
in 11 minutes. Opening both holes would 
cause it to drain in 5 minutes. These were 
exact times, to the second in fact— a re- 
markable achievement considering that all 
his tools were handmade.” 

Liane perked up. “Why should two 
holes be more than twice as fast as one?” 
she asked. 

“The vortex effect, I think,” Ecco re- 
sponded. “Two close holes will create a 
vortex and water will pass through both 
of them faster than the sum of the speeds 
of each hole alone.” 

De Leon nodded in agreement. “What- 
ever the reason, that was the effect. The 
village elders were unhappy with that, be- 
cause they had already invented a full 
moon ritual in which percussion Calabazas 
would be hit every minute as the village 
women chanted. 

“They told the artisan that he must fig- 
ure out how to make a one-minute Cal- 
abaza by the next full moon. He told them 
that he could not, but that with five Cal- 
abazas, four assistants, a ready supply of 
water, and about an hour, he would be 
able to tell the drummers to hit their drums 
every minute exactly.” 

“How long does it take to fill an emp- 
ty Calabaza?” Liane asked. 

“Natasha told me you were clever,” Pan- 
ta said to the 11-year old. “Less than a sec- 
ond. One can just dip it into a big barrel. 
You can assume it takes no time.” 

“Well, then it shouldn’t be so hard,” 
Liane said. “For example, it’s easy enough 
to measure six minutes. You start two Cal- 
abazas at the same time, opening one hole 
of one and two holes of the other. Six 
minutes is the time between the draining 
of the two-hole Calabaza and the drain- 
ing of the one-hole Calabaza.” 

“I think you are on the right track,” de 
Leon said. “But it is important that you 
figure this out today. You see, the full 
moon is about to come and the village ar- 
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tisan has taken terribly ill. They are count- 
ing on me to help them figure out what 
he did. They refuse to let me use my 
watch, as they revere the gourds as holy. 
Now for my personal appeal: As an eth- 
nobiologist, I have made it my goal to win 
their confidence. They trust me so far and 
they have taught me about many won- 
derful herbs. They even have a remark- 
able medicine to enhance musical ability, 
seemingly without side effects, but I must 
stay with them longer before they will 
teach me about it. I would really like to 
help them with their problem first.” 

“Let me try a small example,” Liane said. 
“Suppose that a one-hole Calabaza drained 
in three minutes and a two-hole Calabaza 
drained in two minutes (reversing the vor- 
tex effect for the sake of the example). 
Then it would be possible to measure min- 
utes by allowing Calabaza A to drain in 
three minutes, B to drain in two minutes, 
and then repeatedly filling each Calabaza 
and draining it with two holes. So, A 
would mark times 3, 5, 7, 11, 13,...and B 
would mark times 2, 4, 6, 8, 10, 12,... 
Thus, it would be possible to mark min- 
utes beginning two minutes after the pro- 
cedure starts. Now let’s try it for the gourds 
the artisan really builds.” 

Liane and Ecco talked for a while. Liane 
explained an idea. Ecco smiled. 

“Mr. de Leon,” Ecco said with a smile, 
“Liane has solved your puzzle. Five Cal- 
abazas seems to be the minimum possi- 
ble. If five people follow this simple pro- 
cedure, they will be able to mark single 
minutes starting 40 minutes after begin- 
ning the procedure.” 


Reader: Is five Calabazas really a mini- 
mum? Can you invent a procedure that in- 
volves filling Calabazas and watching them 
drain such that it is possible to mark single 
minutes starting 40 minutes into the proce- 
dure? Assume that it is easy to tell when a 
Calabaza drains. Can you do it better (ei- 
ther with fewer Calabazas or fewer minutes)? 


After Panta de Leon left, Ecco turned to 
me. “Suppose one hole will cause the 
gourd to drain in 7 minutes and two holes 
will cause the gourd to drain in the minute 
below half of 1 (for example, if it takes 
n=20 minutes to drain with one hole, then 
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(continued from page 123) 

two holes requires the integer below 
(20/2)=9 minutes; if it takes n=21 minutes 
with one hole, then with two holes it takes 
the integer below (21/2)=10 minutes). Are 
there any numbers 7 that make marking 
minutes impossible?” 


Reader: Which values of n would be 
troublesome, if any? If none, then map out 
a procedure that can always mark min- 
ules, dssuming you have enough gourds. 


Last Month’s Solution 
Best val: 1801 and deviation for that or- 
dering is: 64 


Object Alice Brad Carla 
27 18 33 61 
14 — 49 35 52 
7 7 6 49 
19 42 13 1 
a 64 1 124 
6 17 10 17 
16 14 ri 36 
5 2 16 0 
2 | 29 o 
23 ip. ? i 
12 3 18 5 
29 34 29 2, 
15 15 22 101 
11 4] 26 5 
13 45 Z2 11 
10 36 2 30 
30 37 12 11 
3 seal 72 62 
24 10 a, 44 
4 9 28 42 
17 34 31 47 
8 56 1 45 
Z1 30 t 20 
22 11 59 oy 
9 80 + a7 
28 7 11 28 
18 | 87 24 
26 12 19 19 
1 val 203 Al 
20 42 158 32 


Deviation of 1 and highest value for this 
deviation is: 1780 


Object Alice — Brad Carla 
6 17 10 17 
18 4 87 24 
2) 30 4 20 
1 71 20 321 
30 37 12 11 
8 56 1 45 
28 7 11 28 
13 45 22 11 
11 41 26 5 
16 14 7 36 
15 15 22 101 
10 36 2 30 
3 131 72 62 
25 64 fl 124 
+ 9 28 42 
14 49 35 52 
29 34 29 3 
24 10 33 44 
26 IZ 19 19 
2 + 29 22 
22 11 Py, 39 
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2/ 18 33 61 
5 2 16 0 
? 80 4 ay 
23 75 2 11 
12 3 18 5 
7 7 6 49 
19 42 13 1 
20 42 158 32 
i 34 31 47 


Reader Notes 

Dr. Ecco’s solution for “Laser Shuttles” 
(DDJ, August 1999) was not the best pos- 
sible. Fortunately, several clever readers 
found better ones. In the best solutions, 
no astronaut will be required to take 
more than three hops to go from site to 
site and the total number of edges is 45. 


e Alan Dragoo’s solution is a nice exam- 
ple chttp://members.aol.com/adragoo/ 
laser.jpg). 

e Burghart Hoffrichter pointed out: “By 
counting the number of horizontal and 
vertical edges in each rectangle and one 
diagonal per rectangle and the sides of 
the triangles, one sees that 45 is the 
maximum number of nonintersecting 
edges.” 

e Brian Nelson thought that deregulation 
might bring about a reduction in the 
number of edges. 

¢ Kees Rijnierse suggested that putting the 
stations slightly off the grid might make 
even better solutions possible, but did 
not know how. 

e Denis Birnie of Transportation & Traffic 
Systems Ltd (Christchurch, New Zealand) 
suggested several other optimization cri- 
teria. Among them: Is there a 4-hop-to- 
anywhere network such that every jour- 
ney can be completed by at least 2 routes 
with only the start and end stations in 
common? If not, is there a 4-hop prima- 
ry and 5-hop secondary network? 


Other readers who demonstrated 45 
edge/3-hop designs included: C. Brock 
Rooney, Pearl Pauling, Rodney Meyer, Pe- 
ter Scholz, Brian Nelson, Tapani Lindgren, 
Todd Ginsberg, Ted Alper, Mike White, Ver- 
non R. Brown, Stephen A. Slezic, Kiia Kallio, 
Tom Rokicki, Matthew D. Ahl, Andy W. De- 
splenter, Mark Taylor, Kees Rijnierse, For- 
est Wilkinson, and Doug Newhort (who 
minimized the average length as well, http:// 
members.home.net/dmewhort/3hub.htm). 

Still others demonstrated 45 edge/4-hop 
designs: James Waldby, Christopher W. 
Curtis, Burghart Hoffrichter, Jason Patter- 
son, Douglas Price, Magne Oestlyngen, 
John Porter, Bob Morton, Greg Smith, Kees 
Rijnierse, Denis Birnie, Carey Bingham, 
Kevin Fox, Steve Bourassa, David Steven- 
son, Jeffrey A. Hallman, and John Grant. 
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A Joke Too Far 


Gregory V. Wilson 


e’ve all been there, I suppose. 
You tell a joke, and everyone 
laughs, so you tell another, and 
another, until suddenly you real- 
ize that you’ve gone a joke too far. Un- 
fortunately, a lot of software is like that 
(Emacs, anyone? Or Windows 2000?)— 
and so are a lot of books. 

A case in point is Mr. Bunny's Big Cup 
0’ Java, by Carlton Egremont III, the se- 
quel to Mr. Bunny’s Guide to ActiveX. The 
latter (reviewed in our April 1999 issue) 





was hilarious, but Mr. Bunny’s Big Cup o’ 


Java falls completely flat, as if the author 
had used up all his good jokes. If you 
have the money to spend, buy a second 
copy of ActiveX to replace the one your 
office mates have “borrowed,” rather than 
Big Cup o’ Java. 

Alan Cooper’s second book, The In- 
mates Are Running the Asylum: Why High- 
Tech Products Drive Us Crazy and How to 
Restore the Sanity, is definitely not “one 
too far’— it makes too many important 
points about why software is hard to use 
for that to be the case. Cooper is the fa- 
ther of Visual Basic, and by all accounts 
(including the ones quoted in the front 
pages of his book) he is a very good user- 
interface designer. I learned a lot from 
About Face, and hoped that this one 
would be equally insightful and icono- 
clastic. 

Well, iconoclastic it certainly is. Take 
the diagram on page 35, for example. It 
shows “power users,” “computer literate 
users,” and “naive users” stacked on top 
of each other, but is captioned “Eu- 
phemism Pyramid.” As the accompanying 
text explains: 


“Naive user” is industry code for “stupid, 
incompetent user.” And certainly these peo- 
ple are made to feel stupid, but they are 


Greg is the author of Practical Parallel Pro- 
gramming (MIT Press, 1995), and coedi- 
tor with Paul Lu of Parallel Programming 
Using C++ (MIT Press, 1996). Greg can 
be reached at guwilson@interlog.com. 
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Carlton Egremont II 





‘Addison Wesley Longman, 1999 
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The Inmates Are 
Running the Asylum 
Alan Cooper 

Sams, 1999 © 
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ISBN 0-672-31649-8 


Readyto in Java 3D 
Kirk Brown and Daniel Petersen 


John Wiley & Sons, 1999 
400 pp., $44.95 
ISBN 0-471-31702-0 


The Java 3D API Specification — 
Henry Sowizral, Kevin Rushforth, 
and Michael Deering 

_ Addison Wesley Longman, 1998 
482 pp., $39.95 

ISBN 0-201-32576-4 


Advanced CORBA 
Programming with C++ 

Michi Henning and Steve Vinsoki 
Addison Wesley Longman, 1999 
1083 pp., $54.95 

ISBN 0-201-37927-9 


Inside the JavaOS 

Operating System 

Tom Saulpaugh and Charles Mirbo 
Addison Wesley Longman, 1999 
184 pp., $29.95 

ISBN 0-201-18393-5 





not. It is the bad design of the interaction 
that is at fault. The apologists in the com- 
puter industry dismiss naive users as not 
significant, but that flies in the face of good 
sense. Why would a vendor write off the 
lion’s share of the market? Because it re- 
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moves the blame for failure from the ex- 
ecutives and software engineers and places 
it squarely onto the shoulders of the inno- 
cent users...The phrase “computer literate 
user” really means the person has been hurt 
so many times that the scar tissue is thick 
enough so he no longer feels the pain. 


Bravo, Mister Cooper— it’s refreshing 
to hear someone finally say that the em- 
peror’s butt is showing. Cooper argues 
that most software is hard to use because 
most engineers are disdainful of the way 
most people think. The only way to fix 
what’s fundamentally broken in so much 
of today’s software is to give interaction 
designers as much real authority in the 
product development process as engi- 
neers. 

Cooper’s points are important, and he 
has a clearer view of our industry and pro- 
fession than most writers. However, I think 
that the book’s occasional glibness and 
one-sidedness weaken his case. For ex- 
ample, the two-dollar psychoanalysis of 
nerds on page 103 might be an accurate 
description of some engineers, but when 
applied to the profession as a whole, it 
becomes the kind of grossly misleading 
generalization that I associate with 
paradigm-shift-per-second marketing gu- 
rus. More troubling is the fact that he nev- 
er discusses how the interaction-centered 
methods he advocates can go wrong. I 
understand how frustration with current 
practices can turn into stridency, but I 
think the major audience for this book 
would find even-handed exposition more 
persuasive than naked advocacy. I don’t 
believe anyone’s method works all of the 
time, and when someone suggests (even 
by omission) that theirs does, it makes 
me suspicious of everything else they’ve 
said. Despite this, I think that Inmates is 
an important book, and I hope that Coop- 
er will turn his hand to writing a text- 
book one day. 

Next in this month’s pile are Ready- 
to-Run Java 3D, by Kirk Brown and 
Daniel Petersen, and The Java 3D API 


1Zy 





Specification, by Henry Sowizral, Kevin 
Rushforth, and Michael Deering. Both 
books are about Java 3D, the 3D graph- 
ics entry in the JavaMedia suite of inter- 
faces. As with the rest of Java, there is lit- 
tle in the library that is actually new. 
Instead, it repackages established tech- 
nology (primarily OpenGL) for use in Java. 
The library includes the usual graphics 
primitives and transformations, along with 
behaviors, handlers for input devices, pick- 
ing, and so on. 

The first major difference between the 
two books is how much context each pro- 
vides. Almost all of The Java 3D API Spec- 
ification is devoted to a method-by- 
method description of the library’s 
contents, while Ready-to-Run spends a 
bit more time showing how those meth- 
ods can be stitched together. I wouldn’t 
recommend trying to find your way 
around the former book unless you are 
already familiar with 3D computer graph- 
ics, but if you are, it’s a handy reference 
and is refreshingly free of typos. 

Which brings us to the other big dif- 
ference between the two books: their 
quality. Ready-to-Run is, quite frankly, 
sloppy. The first figure is unreadable be- 
cause of gratuitous 3D chart junk, ma- 
trices are laid out as ASCII art (honest- 
ly, how hard would it have been to make 
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them look like proper mathematics?), 
Arthur Clarke is misquoted, someone’s 
spellchecker has turned the “Locale” class 
into “Local” sporadically, and some of 
the listings include Sun’s lengthy dis- 
claimers verbatim, along with the source 
code that the reader is supposed to be 
interested in. And do we really need yet 
another slapdash description of how to 
nest applets in HTML documents for dif- 
ferent browsers? 

I really wish Ready-to-Run had been a 
better book, as I don’t think novices will 
be able to make sense of the API Specifi- 
cation. As hardware graphics accelerators 
are getting cheaper, and 3D graphics are 
growing more and more accessible, I hope 
its authors will try again. 

Further down in my pile is Michi Hen- 
ning and Steve Vinoski’s Advanced CORBA 
Programming with C++. I've been waiting 
for this book for a while, and I can hon- 
estly think of nothing to complain about 
except its inordinate fondness for acronyms. 
It is well written, comprehensive, and con- 
tains lots of useful bits of code for its read- 
ers to study and copy. It isn’t particular- 
ly easy going, but that is merely an honest 
reflection of the complexity of its subject. 
The book does not merely describe 
CORBA, it explains it— why certain fea- 
tures were necessary compromises, for 


example, or what errors can be made 
when this or that feature is misused. I 
think this book is going to be as essen- 
tial a part of CORBA programmers’ lives 
as Stevens’s UNIX Network Programming 
has been for everyone who has had to 
wrestle with TCP, UDP, and their ilk. And 
wonder of wonders, its index is actually 
useful. .. 

Finally, there is Inside the JavaOS Op- 
erating System, by Tom Saulpaugh and 
Charles Mirho. Jointly designed by Sun 
and IBM to be a universal platform for 
Java programmers, the JavaOS specifies 
services, device-driver interfaces, memo- 
ry management, interrupts, a microkernel, 
miscellaneous events, and a boot inter- 
face. The book surveys all of these, dis- 
cussing both how they fit together, and 
how they can be used. Except for a cou- 
ple of paragraphs of marketing spiel in 
the introduction, the book is relatively 
hype free (at least by Java standards). If 
JavaOS had taken off, this would proba- 
bly have become the equivalent of Chap- 
pell’s Understanding ActiveX and OLE. 

Now, have you heard the one about the 
three litthe Ethernet hubs? 
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includes nine of the most important books ever written for programmers about the subject. 






|. | These books give you proven and trusted ways to analyze and solve problems in C, C++, and other languages. 
Complete text of nine essential books and 
= = = = Take advantage of the research and testing that went into developing the algorithms and data structures included 
on this CD-ROM! 


Pr fea Es Master practical programming techniques. Ensure that your programs are understandable, maintainable, and 
oes —~ __ «| efficient—even as they grow in size. The right data structure can make your operations simple and efficient, the . 


PRICE! $69. 5 5 wrong one.can make an operation cumbersome and slow. 
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INCLUDES THESE BOOKS: 3. Information Retrieval: Data Structures and 7. Practical Data Structures in C++ by Bryan Flamig 
Chosen by the Editors of DD): Algorithms edited by William B. Frakes and Ricardo 8. Data Structures, Algorithms and Program Style 
1. Data Structures: From Arrays to Priority Queues pace Jats Using C by James F. Korsh and Leonard J. Garrett 
by Wayne Amsbury 4. Fe undamentals of Data Structures 9. Data Structures and Algorithms by Alfred V. Aho, 
2. Introduction to Algorithms by Thomas H. Cormen, 2s cel ics ae lel elk Sc John E. Hopcroft and Jeffrey D. Ullman 
Charles E. Leiserson, and Ronald L. Rivest 5. Reliable Data Structures in C by Thomas Plum Plus more than a dozen articles related to 


6. Data Structures and Algorithm Analysis in C algorithms from Dr. Dobb’s Journal! 


by Mark Allen Weiss 
$29.95 Upgrade available to prior owners! To upgrade call: 800-822-1162 
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Back ie stunts demand! The Alternative Programming Languages Release 3 CD-ROM is your 
definitive resource for cutting-edge programming environments, giving you access to the most 
innovative, solution-providing languages and tools available today. 








LEASE 3 
SLE This new release contains the most comprehensive collection of programming languages ever assembled 
= with over 600 megabytes of data! 


The Alternative Programming Languages Release 3 CD-ROM will get you up to speed with languages and docs for 
Squeak, Dylan, Mozart, Octave, SatherK, Haskell, Lua, EiC, Small Eiffel, Erlang, Modula3, Standard ML of New Jersey 
and more! 


PRICE! $49.9 5 e | | $19.95 Upgrade available to prior owners! To upgrade call: 800-822-1162 | 
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0-992-054 wv te (U.S. & Canada) 


= DrDobb’ All Other Countries: Mail Orders: 


785-841-1631 Dr. Dobb’s CD-ROM Library 
1601 West 23rd St., Suite 200 
E-mail: orders@mfi.com Lawrence, KS 66046-2703 USA 





Fax Orders: 785-841-2624 


Dr. Dobb's Essential Books On Cryptography and.Security CD-ROM 


he editors of Dr. Dobb’s Journal have selected the most important books on encryption technology and have com- 
iled it into a resource every programmer must have. This CD-ROM presents both the theory and practice of net- 
ork security implemented in C, Basic, and other familiar programming languages. 


he Essential Books on Cryptography and Security CD-ROM provides detailed documentation and analysis of: 
lassical and Modern Encryption Algorithms / Private-Key and Public-Key Cryptography / The Data Encryption 
tandard / Signature Schemes / Block Ciphers and Stream Ciphers / Plus Much, Much, More! 
cludes 12 Complete Books! Forward by Bruce Schneier : 
. Applied Cryptology, Cryptographic Protocols, and Computer Security Models by Richard Demillo | 
. Applied Cryptography: Protocols, Algorithms, and Source Code in C, Second Edition by Bruce Schneier PRICE! $99 95 
. Contemporary Cryptology: The Science of Information Integrity edited by Gustavus J. Simmons . 
. Cryptography and Data Security by Dorothy Denning 
. Cryptography: A New Dimension in. Computer Data Security by Carl Meyer 
. Cryptography: Theory and Practice by Douglas Stinson 
Handbook of Applied Cryptography by Paul C. Van Oorschot, Scott A. Vanstone, and Alfred Menezes 
-11 Military Cryptanalysis, Volumes I-IV by William Friedman 
2. RSA Laboratories FAQ on Cryptography, RSA Laboratories Technical Reports, RSA Laboratories Security Bulletins, and CryptoBytes Newsletter 





)r. Dobb's Essential Books On Numerics and Numerical 
rogramming CD-ROM 


)r. Dobb’s Essential Books on Numerics and Numerical Programming CD-ROM is the answer for anyone who needs 


olutions for complex numerical equations. Selected for the value of their content by the editors of DDJ, these books 
n CD-ROM are a must have for everyone - - programmer, engineer, statistician, economist, and scientist. 


cludes these 8 books: 
. Numerical Recipes in C, by William H. Press, Saul A. Teukolsky, William T. Vetterling, and Brian P. Flannery Do oe 
_ Scientific Computing with PCs, by J. C. Nash & Mary M. Nash ecm 
. Fundamentals of Numerical Computing, by L.F. Shampine, R.C. Allen Jr., and S. Pruess a“ 
. Compact Numerical Methods for Computers, Second Edition, by J.C. Nash ) PRICE! $ 99.95 
. Computing for Scientists and Engineers, by William J. Thompson : 
. Accuracy and Stability of Numerical Algorithms, by Nicolas J. Higham 

Numerical Methods: Real-Time and Embedded Systems Programming, by Don Morgan 





. Elementary Numerical Analysis: An Algorithmic Approach, by $.D. Conte and Carl de Boor 


ompiled into easy-to-read PDF format Essential Books on Numerics and Numerical Programming CD-ROM includes these special features: 
dobe Acrobat 3.01 / Fast and accurate full-text search utility 





)r Dobb’s Python Resource CD-ROM mame pene 


Yr. Dobb's Python Resource CD-ROM is the best available source of Python information you'll find! It includes Pytho vm ) 
ython distributions for platforms ranging from Linux to Win32, and utilities and applications for programmers and TV tacuiece 
nd users alike. The CD-ROM also includes tutorials, FAQs, and Python articles from leading magazines such as Resource) 
Mr. Dobb's Journal, Web Techniques, Linux Journal, and more! | one 


hon is a powerful interpreted, object-oriented, and portable scripting language that is easily integrated to 
ll of your programming projects. — : ; 3 


JEW RELEASE! PRICE! $39.95 








)r. Dobh’s Essential Books On File Formats CD-ROM 


his CD-ROM contains the text from six books, providing the most comprehensive and detailed collection of . 
formation on all the important and popular file formats in use today. — 
elected by the editors of Dr. Dobb’s Journal, this CD-ROM contains invaluable information on file formats used 
or graphics, multimedia, sound, databases, spreadsheets, Windows, the internet, and much more! 
o matter what your programming focus, the Essential Books on File Formats CD-ROM discloses the secrets and 
sider knowledge you need to make your programming efforts instantly compatible with all the major 
pplications — right from the start. ; 3 | 

cludes these 6 books: 
he File Formats Handbook by Giinter Born | More File Formats for Popular PC Software by Jeff Walden 

raphics File Formats, 2nd Edition by David C. Kay & John R. Levine / Internet File Formats by Tim Kientzle PRICE! $ 69.95 
ide Windows File Formats by Tom Swan / File Formats by Allen G. Taylor . , 
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Pervasive Software has announced a new 
version of its Tango 2000 web-development 
environment. The visual-development stu- 
dio includes full COM object and Java- 
Bean support, letting you drag-and-drop 
objects directly into their application files. 
Tango 2000 also supports XML and saves 
Tango Application Files in XML format. 
Tango Class Files let you save Tango 
code as objects for reuse in other appli- 
cations. The development environment 
also includes a Web Analyzer for real- 
time monitoring of web-site activity. Tan- 
go 2000 runs on Windows, Macintosh, 
and Solaris. 

Pervasive Software 

12365 Riata Trace Parkway, Building 11 

Austin, TX 78727 

512-231-6000 

http://www.pervasive-sw.com/ 


Parasoft has released Jtest 3.0, an error 
prevention and detection tool for Java de- 
velopment. Jtest 3.0 automatically performs 
static analysis, white-box testing, black- 
box testing, and regression testing. Jtest 
checks code against a set of coding stan- 
dards and offers suggestions on how to 
improve the quality of the code. It also 
analyzes the class bytecodes and auto- 
matically provides a set of inputs, which 
can be customized by you. Jtest 3.0 is cur- 
rently only available for Windows NT, but 
earlier versions run on Solaris and Win- 
dows 98. 

Parasoft 

2031 S. Myrtle Avenue 

Monrovia, CA 91016 

888-305-0041 

http://www.parasoft.com/ 


Scenix Semiconductor has announced the 
SASM SX Cross Assembler, a code devel- 
opment tool that ensures that Virtual Pe- 
ripheral software modules are complete- 
ly reusable within any design environment. 
Virtual Peripheral modules are software 
implementations of peripheral functions 
such as A/D converters, UARTs, Caller ID, 


132 


DTMF generation/detection and other mo- 
dem functions. The new assembler con- 
verts SX symbolic assembly language into 
object code that is readable by the MS- 
DOS operating system, which is the stan- 
dard platform for SX Series development 
and debugging. The SASM assembler, 
which supplements an existing assembler 
from Parallax, Inc. is available from the 
Scenix web site. 

Scenix Semiconductor 

3160 De La Cruz Blvd., #200 

Santa Clara, CA 95054 

408-327-8888 

http://www.scenix.com/ 


Sun Microsystems and IBM have an- 
nounced the Java Media Framework JJMF) 
2.0 API. The JMF 2.0 API features ad- 
vanced media processing capabilities in- 
cluding audio and video capture, com- 
pression and transmission, and support 
for media types and codecs such as MP3, 
Flash, and IBM’s HotMedia. It also con- 
tains an open media architecture to let you 
access and manipulate various compo- 
nents such as effects, tracks, and renders. 
The JMF 2.0 API is available through Sun’s 
Java Developer Connection at http:// 
java.sun.com/jdc/. 

Sun Microsystems 

20450 Stevens Creek Blvd. 

Cupertino, CA 95014 

http://java.sun.com/ 


Softel vdm, Inc. has released SftTree/OCX 
4.0, an ActiveX tree control for Windows 
NT/2000 and Windows 95/98 with sup- 
port for most popular development en- 
vironments such as Visual Basic, Access, 
Delphi, and C++ Builder. The new re- 
lease includes support for virtual lists, a 
vertical splitter bar between columns, OLE 
drag-and-drop, digitally signed cabinet 
files, multicolumn support with cell col- 
ors, a background bitmap, and item 
searching. SftTree/OCX offers hierarchi- 
cal data displays with advanced features 
such as multiple columns. SftTree/OCX 
4.0 is offered at $299.00 for a single de- 
veloper license. 

Softel vdm, Inc. 

318 Tamiami Trail, Suite 14, 

Punta Gorda, FL 33950 

941-505-8600 

http://www.softelvdm.com/ 


Queue Systems has announced Q-Mes- 
sage, general-purpose messaging soft- 
ware that provides the ability to com- 
municate between applications running 
on the same platform, applications run- 
ning on different servers, and applica- 
tions running on a client-server archi- 
tecture. Q-Message features a singular 
interface regardless of the platform, asyn- 
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chronous programming, an object inter- 
face, and does not require socket pro- 
gramming. Q-Message is available for the 
Open VMS, UNIX, and Windows NT plat- 
forms. A server license costs $5000 and 
$500.00 per client workstation. 

Queue Systems 

1800 Saint Julian Place 

Columbia, SC 29204 

803-771-4100 

http://www .queuesys.com/ 


Object Domain Systems has released Ob- 
jectDomain 2.5, a UML CASE tool. Fea- 
tures of ObjectDomain 2.5 include sup- 
port for all nine OMG UML diagrams 
backed by a true UML meta model; 
smaller memory footprint; version con- 
trol and multiuser support via file-based 
repository; HTML documentation and 
glossary generation for user models; 
model searching with regular expression 
support, and full round-trip for Java. Ob- 
jectDomain 2.x will run on any platform 
that provides a Java Runtime Environ- 
ment, and supports code generation and 
reverse engineering for Java, C++, and 
Python. Existing features also include di- 
agramming with auto-layout, Bezier 
curves, color support, stereotype icons, 
and extensive control of presentation as- 
pects. The price of ObjectDomain Z2 


professional addition is $995.00 and the - 


standard edition is $495.00. 
Object Domain Systems 
1130 Situs Court, Suite 250 
Raleigh, NC 27606 
919-573-7588 
http://www.objectdomain.com/ 


Boxer Software has released Boxer 99, a 
text editor for Windows NT/98/95. Boxer 
99 includes color syntax highlighting, a 
feature that displays and prints source 
code for popular programming languages 
in user-defined colors. Users can modify 
the reserved words and syntax, allowing 
Boxer to display any language in any com- 
bination of colors and font styles. Boxer 
99 lets you open an unlimited number of 
files, and supports file sizes up to 16 MB 
and line lengths to 32 KB. Boxer 99 is 
priced at $59.00 and may be purchased 
online. Network and multiuser licenses are 
also available. 

Boxer Software 

P.O. Box 14545 

Scottsdale, AZ 85267-4545 

800-982-6937 

http://www.boxersoftware.com/ 


TextPad 4.0 from Helios Software Solu- 
tions is a Windows 98/95/NT text editor 
that supports color syntax highlighting 
for C/C++, HTML, and Java, as well as free 

(continued on page 135) 
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Dr. Dobb’s Essential 
Books on Numerics and 
Numerical Programming 
CD-ROM is the answer for anyone 
who needs solutions to complex numerical 
equations. Selected for the value of their 
content by the editors of DDJ, these books on 
CD-ROM are a must have for all programmers. 
The Essential Books on Numerics and Numerical Programming CD-ROM 
gives you the power to search quickly and accurately across the complete text 
of all eight books. 





COMPUTING 
FOR 
SCIENTISTS 
AND 
ENGINEERS 


NUMERICAL ANALYSIS 


An Algorithmic Approach 





Accuracy and Stability of Numerical Fundamentals of Numerical Computing, Numerical Recipes in C, by William H. Press, 
Algorithms, by Nicolas J. Higham by L.F. Shampine, R.C. Allen Jr., and S. Pruess Saul A. Teukolsky, William T. Vetterling, and 
Compact Numerical Methods for Computers, _Numencal Methods: Real-Time and Brian P. Flannery 

Second Edition, by J.C. Nash Embedded Systems Programming Scientific Computing with PCs, 
Computing for Scientists and Engineers, by Don Morgan by John C. Nash 

by William J. Thompson Elementary Numerical Analysis: An 


Algorithmic Approach by S.D. Conte 
and Carl de Boor 


This comprehensive reference tool will give you: 


Detailed introduction to numerics, numerical analysis, and their applications 


* Solutions to numerical problems based on limited memory storage space for data 
and applications 


Thorough up-to-date treatment of the behavior of numerical algorithms in finite, 
precision arithmetic, combining algorithmic derivations, and rounding analysis 


* Positional number theory, fixed-and floating-point arithmetic 
* Multigrid methods for solving partial differential equations 
* Spectral analysis on unevenly sampled data 


* Calculation of numerical derivatives 





| * Write efficient, high-speed arithmetic routines, and more! 


Go to our web site for all the products 


from Dr. Dobb’s CD-ROM Library: www.ddj.com/ cdrom/ 
Order Tod ay! All Other Countries: Mail Orders: Dr: Dobb's 


785-841-1631 Dr. Dobb’s CD-ROM Library 
| 800-992-0549 E-mail: orders @ mfi.com 1601 West 23rd St., Suite 200 


| U.S. & Canada Fax Orders: 785-841-2624 Lawrence, KS 66046-2703 USA 











“Your Product Contains 
Gaping Security Holes” 


-- Bruce Schneier, Contributing Editoi 


The kind of press coverage you want to avoid, right? Preventinc 
this type of exposure requires extensive planning throughout 
the entire development process. So where do you turn with 

Price: $99.95 questions? A trusted source. 





Text from 12 books 


Dr. Dobb’s Essential Books on Cryptography and Security CD-ROM 













ecurity and cryptography are : Includes the following 
among the most important and : books selected by 
complex challenges programmers face in : the editors of 
today’s networked world. That's why the: Dr. Dobb's Journal: 
editors of Dr. Dobb’s Journal have selected : 1. Applied Cryptography: Protocols, 

: Algorithms, and Source Code in C, 
and put on CD-ROM twelve essential : Second Edition, by Bruce Schneier 
books all programmers should have : . 

: 2. Cryptography: Theory and Practice, 


cryptographic and security experts such aS: 2 pondpook of Applied Cryptography, 


William Friedman, Bruce Schneier, Dorothy by Paul van Oorschot, Scott Vanston, 
: and Alfred Menezes 


The Essential Books on Cryptography 
and Security CD-ROM provides detailed 
documentation and analysis of: 


e Classical and Modern e Generation, Distribution, 

Encryption Algorithms and Installation of 
Cryptographic Keys 

e Private-Key and Public- 

Key Cryptography e Communication 
Architecture Encryption 

e The Data Encryption 

Standard 


Authentication Techniques 


Denning, Alfred Menezes, and others, : ° Signature Schemes e Applying Cryptography 

: 4. Cryptography and Data Security, (Digital Signature | to Pin-Based Electronic 
Pele Resen tals oni ge : by Dorothy Denning Standard) | Funds Transfer Systems 
Cryptography and Security CD-ROM : ee eee — : 
“itt ; : 2. Ulyplograpny: A vew Uimension in e Approaches to Key e Measures of Secrecy for 
Is the Benne ieee of cryptographic : Computer Data Security, by Carl Distribution and Key Cryptographic Systems 
and security information. : Meyer and Stephen Matyas Agreement Protocols 

: B Anaied-Cwal ; e Access, Inference, and 

From algorithms for random numbers ‘by Ric ae aa ig e Block Ciphers and Information Controls 

; Stream Ciphers 
and hashing, idee Oy coniie== 0) and : 7 Contemporary Cryptology, edited © Plus Much, Much, More! 
Monte Carlo simulations, Dr. Dobb's ; by Gustavus Simmons 
Essential Books on Cryptography and : 8-11 Military Cryptanalysis, 
Security CD-ROM presents both the : Volumes I-IV, by William Friedman 
theory and practice of network security : 12. Basic Cryptanalysis, by U.S. Army 
familiar programming languages. : by RSA Data Security 





Add this CD-ROM to your digital bookshelf: 
and possess the single most comprehen- 
Sive resource of encryption information 
available. 








(continued from page 132) 
add-ons for Perl, AutoLisp, and Pascal. The 
size of your documents is limited only 
by the amount of memory available, and 
you can edit multiple open documents 
simultaneously. You can create document 
classes, associate file names with them, 
and define different parameters for each 
document class. Combinations of com- 
mands can be saved as keystroke macros 
and replayed with hot keys. TextPad sup- 
ports fixed and proportional fonts, as 
well as double-byte character sets for 
Asian languages. You can view binary 
files in hexadecimal format. TextPad 4.0 
costs $27.00 and can be purchased on- 
line at http://www.textpad.com/. 

Helios Software Solutions 

P.O. Box 619 

Longridge, PR3 2 GW England 

44-1772-786373 

http://www.textpad.com/ 


TurboLinux has announced TurboCluster 
Server, which lets users cluster servers run- 
ning Sun Solaris and Microsoft Windows 
NT with Linux. TurboCluster Server runs 
on open standard Intel and Alpha archi- 
tecture platforms. It seamlessly integrates 
into existing networks and virtualizes an 
IP service by providing a single IP and 
hostname for a group of servers. These 
servers each run their own independent 
copies of the clustered application. Tur- 
boCluster Server ships with a suggested 
list price of $995.00 for two nodes and 
$1995 for three nodes or more. 

TurboLinux 

2000 Sierra Point Parkway, Suite 702 

Brisbane, CA 94005 

650-244-7777 

http://www.turbolinux.com/ 


QNX Software Systems and the Espial 
Group have announced a PersonalJava 
solution for set-top boxes, web phones, 
PDAs, handheld computers, and a vari- 
ety of other network devices. The tool 
includes a comprehensive suite of visu- 
al design tools, user-interface compo- 
nents, platform products, product APIs, 
PIM capabilities, and Internet applica- 
tions. The applications include Espial Es- 
cape, a full-featured web browser with 
a core footprint of less than 175 KB; 
Ebox, an e-mail client; and Espial Assis- 
tant, a complete PIM with an address 
book, calendar, and appointment man- 
ager. The browser adds support for 
JavaScript, SSL, applets, and printing. The 
Espial product suite includes DeviceTop, 
which incorporates an application 
launcher, support for remote software 
upgrades and downloads, input device 
management, virtual keyboards, a mul- 
tiple application environment, and 
user/data rights management. Espial also 


http:/)www.ddj.com 


provides the visual design tools Kalos 
Espresso and Kalos Architect. OEM pric- 
ing is available from both Espial Group 
and QNX Software. 
QNX Software Systems Ltd. 
175 Terence Matthews Crescent 
Kanata, Ontario, K2M 1W8, Canada 
800-676-0566 


http://www.qnx.com/ 


Espial Group 

200 Elgin Street, Suite 205 
Ottawa, Ontario, 

K2P OP7 Canada 
800-437-7425 
http://www.espial.com/ 


Code Fusion from Cygnus Solutions com- 
bines Cygnus- certified open-source GNU 
tools with a graphical IDE framework. The 
Code Fusion IDE features a C, C++, and 
Java tools product manager, editor, graph- 
ical browsers, and the Cygnus Insight de- 
bugger. Cygnus Code Fusion supports all 
major Linux distributions. Code Fusion 
IDE is priced at $299.00 and is immedi- 
ately available for online purchase at 
http://www.cygnus.com/codefusion/. 

Cygnus Solutions 

1325 Chesapeake Terrace 

Sunnyvale, CA 94089 

800-294-6871 

http://www.cygnus.com/ 


Lineo (formerly Caldera Thin Clients) has. 


begun shipping an embedded micro- 
browser product, called Embrowser, 
which lets you create a customizable In- 
ternet browsing interface for embedded 
devices such as set-top boxes. Targeted 
at embedded OEMs, Embrowser is in- 
tended to be paired with customized 
hardware, HTML interfaces and back- 
end content designed by the OEM cus- 
tomer. Embrowser currently runs on DR 
DOS, and a version will also be avail- 
able on Embedix, Lineo’s embedded ver- 
sion of Linux. 

Lineo, Inc. 

383 South 520 West 

Lindon, UT 84042 

801-426-5001 

http://www .lineo.com/ 


WoodenChair Software has announced a 
new suite of namespace management and 
Javadoc authoring and editing tools. The 
Utility+ Lite edition supports major inte- 
grated development environments (IDEs) 
such as VisualCafe, VisualAge, and 
JBuilder. The namespace management tool, 
Repackager+ Lite, handles package struc- 
tures, import statements and class names. 
It updates the IDE project file (*.jpr or 
* vep) and all files in the project to reflect 
the new class name chosen. The Doct 
authoring/editing tool for javadoc docu- 
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mentation allows you to create, remove 
and edit javadoc tags. Features include 
auto-format of Javadoc, comment con- 
version of existing code, autofill of cer- 
tain tags from source, creation of custom 
javadoc tags and application of templates 
created with Doct. PackageMaps created 
with RePackager+ can update dependent 
source in your projects, and Swing Pack- 
ageMaps is included to update your source 
to the new javax.swing.* namespace. A 
standalone version for working with the 
JDK and text-based editors is also avail- 
able. Free evaluation copies are available 
from the company’s web site. 

Woodenchair Software Corp. 

8 Kemsing Court 

Markham, Ontario 

L3R 4P7 Canada 

905-479-2243 

http://www.woodenchair.com/ 


BrainTech has released ODYSEE Devel- 
opment Studio 3.0, a drag-and-drop en- 
vironment for application development. 
ODYSEE features an integrated environ- 
ment with GUI builder; online debugging 
and optimization, ODYSEE-wrap for in- 
tegrating C/C++ functions, and a compo- 
nent-based architecture. Such an open ar- 
chitecture has no limits in the type and 
number of components that can be 
brought into its environment. A free trial 
version of ODYSEE Studio is available on- 
line at http://www.bnti.com/. 

BrainTech Inc. 

Unit #102 - 930 West Ist Stre 

North Vancouver, BC 

V7P 3N4 Canada 

604-988-6440 

http://www.bnti.com/ 


Quality Systems and Software (QSS) has 
released DOORSConnect TestDirector, 
which links QSS DOORS with Mercury In- 
teractive’s TestDirector, allowing software 
developers to capture end-user require- 
ments and trace them from initial design 
through final testing. The new tool links 
the requirements and testing processes. 
DOORS and TestDirector are synchro- 
nized, with a color-coded DOORS view 
showing the current status of TestDirec- 
tor tests and defects. DOORS filters can 
be used to generate various reports that 
show, for example, only tests that have 
passed or failed. The package runs on the 
Windows platform and is available for a 
single-user license price of $500. 

Quality Systems & Software 

200 Valley Rd., #306 

Mt. Arlington, NJ 07856 

973-770-6400 

http://www.qssinc.com/ 


Esker has announced availability of the Al- 
com LanFax SDK, which lets you embed 
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fax automation directly into applications. 
The SDK lets you integrate Alcom Lan- 
Fax NT with other applications to add 
automated fax routines into databases, 
transaction processing systems, docu- 
ment management, imaging, and work- 
flow applications. The SDK is accessible 
using Visual Basic, C/C++, and Visual 
C++, as well as scripting languages such 
as VB Script. This library exposes the 
programmable fax messaging objects in- 
cluding Alcom LanFax NT phonebook 
information, fax messages, destination 
information, attachments, and other fax 
messaging components. The Alcom Lan- 
Fax SDK is free and is available via 
download from http://www.alcom.com/. 

Esker Inc. 

100 E. 7th Avenue 

Stillwater, OK 74074 

405-624-8000 

http://www.esker.com/ 


Motorola and emWare have jointly re- 
leased the NET.08 SDK. Based on Mo- 
torola DigitalDNA’s 8-bit 6SHC908GP32 
Flash microcontroller, the NET.08 kit con- 
tains a development environment for em- 
bedding network connectivity into micro- 
controller-based devices Uike power 
meters) and building controls and home 
automation devices without requiring an 
RTOS or TCP/IP stack in the device. 
Flash memory on the chip provides flex- 
ibility by allowing devices to be de- 
bugged, enhanced, or upgraded remotely 
through the network. The NET.08 soft- 
ware development kit is available for 
$99.00 from emStore (http://www.emstore 
.com/motorola/) and contains: Version 
3.0 of emWare’s EMIT (Embedded Micro 
Internetworking Technology) device- 
networking SDK; Netscape and Microsoft 
browsers; Explorer 4.0; Standard emMi- 
cro port to Motorola’s 68HCO8 in C 
source code; emWare hex file used in 
programming the 68HC08-based device; 
online general EMIT user’s and devel- 
oper’s manuals and user-interface man- 
ual; demo applications; online NET.08 
specific developer’s guide and reference 
board manuals; Motorola microcontroller 
reference manuals (on CD-ROM); the 
reference board including Motorola 
68HC908GP32 Flash microcontroller; 9V 
battery with cable; and an RS232-to- 
RS232 device cable. 

emWare Inc. 

6322 South 3000 East, Suite 250 

Salt Lake City, UT 84121 

801-453-9300 

http://www.emware.com/ 


VMware has announced the release of 
VMware for Windows NT and Windows 
2000. VMware enables PCs to run one or 
more protected sessions concurrently us- 
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ing one or more operating systems on a 
single machine. VMware supports a vari- 
ety of native and Virtual Machine operat- 
ing systems, including Linux, FreeBSD, 
NetBSD, and other BSD versions of UNIX 
and Solaris for Intel processors, as well as 
Windows 95/98. VMware does not mod- 
ify the host operating system and does 
not require rebooting, repartitioning, or 
reconfiguring the system. It provides a 
graphical interface for flipping between 
applications or operating systems running 
in the background. Users can encapsu- 
late their PC environments and run them 
on any machine with a simple save and 
restart procedure. A Pentium processor 
running at 266 Mhz or faster and a mini- 
mum of 96 MB of RAM is recommended. 

VMware Inc. 

3145 Porter Drive, Building F 

Palo Alto, CA 94304 

650-475-5000 


http://www.vmware.com/ 


KLGroup has released Jprobe 2.5 Server- 
side Edition. The Jprobe suite of devel- 
opment tools offers Solaris support, per- 
formance tuning capabilities, memory 
debugging,and thread analysis of Enter- 
prise JavaBeans (EJBs) and servlets run- 
ning under popular application and web 
servers. Jprobe 2.5 also supports IBM Vi- 
sual Age for Java. The tool includes a 
new web and application integration fea- 
ture called Server Launch Pad, which au- 
tomatically configures Jprobe to work 
with various application servers, either 
locally or remotely. The Launch Pad cur- 
rently includes support for IBM Web- 
Sphere, NEA WebLogic Server, Java Web 
Server, Allaire/Live Software Jrun, Sun 
Servlet Runner, and user-configured Java 
servers. Pricing for Jprobe 2.5 starts at 
$499.00. 

KLGroup 

260 King Street East 

Toronto, Ontario 

M5A 4L5 Canada 

416-594-1026 

http://www.klgroup.com/ 


AuthenTec has announced availability of 
an Evaluation Kit and a SDK that make 
it possible for you to develop silicon- 
based, subsurface fingerprint identifica- 
tion software. The kits include evalua- 
tion copies of the company’s FingerLoc 
Personal Identification System, based on 
its E-Field Technology fingerprint iden- 
tification technology that reads beneath 
the surface layer of the skin to the live 
layer, where the true fingerprint begins. 
E-Field Technology is not dependent on 
a finger’s surface features, which are con- 
stantly affected by changing external fac- 
tors such as dirt, calluses, and dryness. 
The Evaluation Kit includes the Finger- 
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Loc AF-S1 System along with its biometric 
evaluation program, which provides en- 
rollment and matching software in both 
verification and identification modes. 
Presently, the Evaluation Kit is offered 
as a standalone PC peripheral with an 
RS-232 serial interface. The company also 
plans to release a USB version. The SDK 
upgrade includes the evaluation kit; de- 
velopment software with APIs; user’s 
guide and programmer’s reference man- 
ual; and a Windows-based demo for log- 
in, file encryption, and password bank. 
The Evaluation Kit is priced at $995.00 
and the SDK is priced at $4595.00. Ad- 
ditional sensors may be obtained for $125 
in quantities of 1-10 and for $100 in 
quantities of 11-100. 

AuthenTec, Inc. 

P.O. Box 2719 

Melbourne, FL 32902-2719 

407-308-1300 

http://www.authentec.com/ 


Packet Technologies has announced 
Sheriff On Board, a firewall on a single 
chip for application in 10/100 Base-T, PC- 
LAN NIC, and USB-based networks. The 
built-in dedicated CPU enables Sheriff 
On Board to provide application screen- 
ing, protection against spoofing, real- 
time scanning of intruders, static filter- 
ing, stateful inspection, packet data 
inspection, network address translation 
(NAT), and accurate logging and report- 
ing. All incoming packets are parallel 
processed and analyzed for legitimacy. 
Processing for both incoming and out- 
going packets operates at network 
speeds of up to 100 Mbits/sec. Sheriff 
On Board incorporates a web-browser 
management interface. The system is 
available in three core versions: a 10/100 
base-T to 10/100 base-T firewall on a sin- 
gle chip targeting the cable modem and 
xDSL modem market, a PCI-LAN NIC 
firewall on a single chip targeting the 
mass production market (replacing the 
standard NIC on the corporate PC and 
providing LAN connection and a high- 
level policy-based security solution), and 
a 10/100 base-T to USB firewall on a sin- 
gle chip targeting new products incor- 
porating the USB interface. Sheriff On 
Board is based on the company’s VLSI- 
based Reduced Instruction Security Pro- 
cessor. 

Packet Technologies Ltd. 

6 Hamachtesh Street 

Holon 58810 Israel 

972-3-558 7001 

http://www.packet-technologies.com/ 
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Microsoft: [eeaiee 


Solution Provider 


Master C++, Visual C++ & COM in Minutes 


Get Real-World C++ Training From Hany Greiss 


Dr. Hany Greiss has over 6 years experience as a Senior Technical Instructor in Oracle, UNIX, 
C++, Windows, Java, Tel, & SQL for PEAC inc, in Ottawa, Ontario. In this course he brings to 
you his expertise in the speafic area of PL/SQL for Oracle 7. Dr. Greiss has 16 years of extensive 
Knowledge as a software developer, senior analyst and consultant. He knows his stuff! 


COM/Activex 


6 Videos, 6 Workbooks 
Over 12 Hours of Training 


4pVad 















Course Price 
COM Level 1- Fundamentals of OOP $89.95 
COM Level 2- Fundamentals of COM $89.95 
COM Level 3- COM Features $89.95 
COM Level 4- COM Featrs. DLLs & Registry $89.95 
COM Level 5 $89.95 
COM Level 6 $89.95 





Core Concepts of Object Oriented Programming » Core COM Conce 
Unknown Interface « Unknown Details * Reference Counting * Stand-Alone Components 
Reference Count Details * Multi-Component DLL's « The Registry * DLL Self Registration 
Client Using DLL * Component Factories * Implementing Class Factory * Writing DLL's 
Support IClass Factory * Automation (IDispatch Interface) * Tools * Smart Pointers 
Off-the-Shelf Smart Pointers * Wrapper Classes * Server-Side Tools « Basic Principles 
Automation Servers * Automation Controllers * Active Template * Using COM Objects 
Automating PowerPoint * MFC Component Creation 
Building a Component in MFC 
















DEVELOPING COM 
ACTIVEX COMPONENTS 


LEVEL | FUNDAMENTALS 
OF OBSECT ORIENTED PROGRAMMING 


ag 





senna SN 






We accept C.0.0., Company PO. or check + v=) & C 


AS Corp.@, WordPerfect®, MS and Microsoft®, Lotus@, Borland@, Aldus@ and their products are trademarks and/or registered trademarks 
if Novell Inc., Microsoft Corp,, Lotus Development Corp, Borland International Inc., and Aldus Corp,, respectively. 


pts* COM Hands-On 





Visual Basic 6.0 


17 Videos or CDs, 17 Workbooks 
Over 34 Hours of Training 


aplud 





Course Price 

VB 6.0 Getting Started $599.95 
VB 6.0 Power Developer $699.95 
VB 6.0 Component Development $89.95 


Over 


Tittes 


Hany Greiss 
COM/ActiveX 
Instructor 
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® Building Linux Servers 

e C++ Programming 

® Component-Based Development for Distributed Systems 
® COM&/Windows 2000 

e Enterprise Application Integration 

e Java Programming 

® Methods & Modeling 

® User Interface Design 

* XML Development 


Also featuring our 2-day Management Conference 


Visionary Keynotes sponsored by #4 


WORLD 


OPENING KEYNOTE: 
Wednesday, November 10, 10-11lam 





| JAVA & JINI: TOWARDS A SCIENCE OF PROGRAMMING 
Thursday, November 11, 10-11lam 


ESCAPING ENTROPY DEATH: THE IMPERATIVE OF XML & JAVA 


Special Events & Parties 

Focus on hot-button issues and bring tough questions to our 
expert developers during our special events. Then let off 
steam at our FUN evening parties. 


e Roundtable Discussions 
e Design Clinics in Java and C++ 
Power Panels | | 
e Birds-of-a-Feather Parties for COM, XML and Java users 
e SD'99 Gig & Bites Party 
e “You Have the Floor” Party 
e Developalooza ‘99 Party 
e Career Development Sessions 


( Technical Sessions by (GOMPUTER® s onal 





fying software teams 


iSSOCIATES 


Xm Re Software superior by design. 


ie ol ¢ 
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The year's most anticipated showcase features the full 
spectrum of development tools, where vendors send their 
engineers to interact with you. Product launches, free 
technical seminars and a non-stop series of special events 
are all part of the action. 


e Targeted Pavilions: IBM Partner Pavilion; XML 
Pavilion; Distributed Object Computing Pavilion 

* Online Lounge: Play networked games or simply network 
with your peers! 


e Exhibitors Include: 


AD Experts LearningPatterns.com Rogue Wave 

Alis mY fod OF-] oY -) Seagull 

Computer Associates Object Design Sourcecoffsite 
Compuware Parasoft StarBase 

ESRI Prentice Hall Teamshare 

IBM Princeton Softech W=Xod plate) (ele hVam > Count-lale (=. 
Insight etc Tia) exe) am K-Yod alave) (ele nv Template Software 


Intraware Rational Software Wiz Art Software 





Miller Freeman 


Register for the 


and receivea 


LIMITED EDITION 


Fleece Jacket! 













Sharpen Your 
Internet Focus at 
SD|99 — 


www.sdexpo.com { ‘2 CONFERENCE NOVEMBER 8—12 
: CONTACT US FOR EXPO NOVEMBER 9—11 , ; 


REGISTRATION INFORMATION SDI99 WASHINGTON CONVENTION 


415 905 270c*sdij@mfiicom  —_ software WASHINGTON D.C. 



















SofiAare Copceper. 


One Solution for 
PC and Mac? 


Yes, the WIBU-KEY . 
USB hardware is the same / 
for PC and Macintosh. Network 
Program the hardware Features, 
at the PC and deliver __ eee 
it with a Macintosh Ceatures. 
version of software. license 
' 
No problem! Manager, 
P Year 2000 
All with one system. 
. Guarantee, 
All with WIBU-KEY! ’ 
Best Service 
FA. 
Order Your Test Kit [773 
Hall A2 - Booth 533 "41:10 310). @ 
International: 
WIL <VSTEMERE 
F — Rueppurrer Strasse 52-54 < 
= D-76137 Karlsruhe ~w TECHNOLOGIES 
tel.: +49-721-93172-0 USA, Canada: Griffin Technologies, LLC é 
= fax: +49-721-93172-22 tel.: (785) 832-2070 - fax.: (785) 832-8787 be 
Quality the World Trusts — email: info@wibu.com email: sales@griftech.com - www.griftech.com % Ng 
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Need high speed ?... check out 


our fast comm cards ! 


A. 


www.wibu.com 





T1/E1, Fractional T1, T3/ 
E3, ISDN, & HSSI speeds. 


You'll be able to run 

Frame Relay, ATM, X.25, 
PPP, LAP-B, HDLC/SDLC, 
SNA & custom protocols. 


fox 805-966-7630 4%& Quadron’ 


telephone 805-966-6424 “a 


email info@quadron.com WWW. ¢ uadron.com 





JOURNAL 


Consulting « Mentoring ¢ Training ¢ Placements 


Sia SYSNETICS 


Object Oriented Technologists 


Sysnetics™ specializes in Object Oriented (00) 
Technology for software architecture, design and 
development of Distributed Systems. 


Based on our exceptionally high standards, people and 
methods, SYSNETICS™ helps managers minimize risk in the 
areas of people resources, technical design, architecture 
and implementation for large-scale OO projects. 


™ 00 Technology: Java," C++, Visual C++, Smalltalk 
™ Distributed Technology: CORBA, COM/DCOM 


™ 00 Methods and Tools: UML, Booch, OMT, 
Rational Rose, Paradigm Plus 


SYSNETICS"” 

150 153rd Avenue, Suite 300B 
Madeira Beach, FL 33708 
v: 800-270-6099 
e-mail: info@sysnetics.com 
www.sysnetics.com 


Program with 
pictures, not code! 


Use with ACT!, Word, 
Excel, Powerpoint, or 
any COM-enabled 
application! 


= 
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Sanscript ™- by Northwoods Software 














—7 Fortune 500 companies .. 


D TE Edit Control (Advanced RTF control) 
BD HTML Viewer/Editor Add-on for TE 

D ReportEase Plus (report writer engine) 
D SpellTime DLL and dictionary 

D FormPlus (form designer/filler) 

D Rich Text Grid control and ChartPro 


_ Demos: www.subsystems.com 


ALL PRODUCTS 


INCLUDE COMPLETE 


SOURCE CODE, 


ARE ROYALTY FREE 
AND AVAILABLE FOR 
lb OR 32 BITS. 


SUB SYSTEMS, INC. 
1] Tiger Row, Georgetown, MA 01833 
978-352-9020 Fax: 978-352-9019 


mime ++ 
“a most complete and essential class library” 


Licensed by ... and around 


: the world! 
|) \ document object model for MIME 
i C++ library 


- / fully standards compliant (RFC 822, 2045, 2046, & more) . 


{ SMTP, POP, NNTP 

V source code available 
Hunny Software ¢ (301) 948-6999 
www.hunnysoft.com/MIMEPP 


© 
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Conventional (++ arithmetic can result in unacceptable 
rounding errors and even destructive, high-order truncation. 
Now, you can easily replace numeric datatype declarations 
(double, int, unsigned long, etc.) with our new patent: 
pending Base/1 Number Class and improve the accuracy 
of calculations in any C++ application. 


Base/1 Number Class gives you: 


e Uncompromisingly exact © Efficient arithmetic and 
decimal arithmetic compact representation 


© Up to 100 significant e Fully documented ANSI 
digits of precision with (++ source code and 
decimal point at any examples 
desired digit 
Guarantee numeric precision in 
all your applications. 


Visit www.boic.com or call 212-691-7155 to order 
your copy now for only $199. 


Base One International Corporation 
E-mail info@boic.com @ Visit www.boic.com 
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The best cross-platform 
compression libraries 
for Win32, Win16, 
DOS, OS/2, Unix, 
Macintosh, and 
embedded systems. 


Robust 45-function API compresses 


buffers, files, archives, disk spanning, § 


encryption, self-extr. EXE's and more. 


FREE DEMO Call 1-800-775-1073 


Tel (678) 442-1623 


The Right Choice ae 


- Scientific, Engineering 
and Industrial software 
development worldwide. 
-10 year history of successful 
corporate partnering. 
- A Philadelphia 100 company. 


Scientific Software Tools, Inc. 
610-891-1640 ¢ info@sstnet.com 





- REE | voursel from the stress, 
Strain and headaches 
that can accompany 
custom programming 
worries! 

NAG Numerical 
Libraries & 
Visualization tools 


NT Windows C/C++ 
UNIX SMP 
Solaris Parallel 
Linux Fortran 


'/¢ Increase productivity * Reduce costs 
| ° Accelerate development ¢ Cutting edge 


algorithms ¢ Quality documentation 


1 Email NAG today at info-ddj @nag.com 


and we will send you information for obtaining your 


| Free NAG Worrybeads and information on quali- 


ty, dependable NAG products to relieve your 
numeric headaches. 


_ ©) NUMERICAL ALGORITHMS GROUP e 
. _ 630-971-2337 
— DC Micro Fox (678) 442-1819 |. : 

| == Development —— www.demicro.com 


NEW! 


Code Co-op... Version 2.1 


The versatile Version Control System for 
collaborative development 
® Synchronization using email, local network, floppy disk 


® Intuitive GUI -- check-in, check-out, synch, visual diff. 
® Fully functional trial version available for download 


 SERVER-LESS VERSION CONTROL | 


www.relisoft.com 





Reliable Software, fj 
— Smart Tools for Smart Programmers. 

































GUD 
Create Prolog = 
Components = 


Diagnose, advise, configure and plan 
with the Amzi!® LogicServer™ tools & 
libraries (DLLs) for C/C-+#, Java, VB, 
Delphi, Web Servers & more. Win NT 
95 3.x & Solaris. Use ODBC, Sockets, 
Unicode & new OOP extensions. 


hy pun 5s wae % “= g 
FREE Evaluation Version! 























Produce HTML, 
MSHelp, and 
MSWord 
documentation from 
comments in your 
code - and you won’t : 
need to change your 
commenting style! 


Scans, stores & indexes Images for instant access 

Creates Objects Folders keyed to your Account #’s 
Supports a full range of file types including TIFF, JPEG, GIF, 
PDF, DOC, XLS, AVI, WAV plus many more 

Includes a Web Object Server to any Browser 

Quickly integrated using ActiveX, DDE, Program CALL, 
Designer Wizard, or Unix & Mainframe “Screen Scraping” 
Scalable, multi-user, network-enabled, Win95/98/NT 






You can fine- 
tune your output 
» with DocJet’s 
WYSIWYG 


























http://www.tall-tree.com 
info@tall-tree.com 512-453-4909 






: Email info@amzi.com ' 
Cat +1-513-425-8050 Fax 425-8025 | 









» New -Version 2.1. 


» Generates documen- 
tation. directly from the 
source code. 


Pm Extracts comments. 
p> User customized 
reports formats. 
& HTML, WinHelp, 
RTF. 





SECURITY 







@ Improve OCR accuracy by 50% or 


more. 1,000,000+ users w orldwide 
suze Forms ID and extraction. Allows OCR/ 
p FREE working orm ix ICR to work well on difficult forms 
evaluation at = see Fast color extraction for OCR 
www.bbeesoft.com pectrum and more. AIIM show winner 
The original tool for viewing 
ViewDirector and deem 
p, fast and robust 
Demos and information 


at 
800-867-5250 


CrypKey is solid, software-only copy protection with a 
broad range of features to control all aspects of your 
product’s operation. You can offer free one-time trials, turn 
demos into full multi-user network versions, or sell specific 
options by phone, fax or email. Sell usage by time, runs, 
features - the control is all there - easy to implement, and 
even easier to support; and it’s not fooled by reinstallation 
or back-dating. Imagine Internet distribution of your 
software with easy e-commerce features and automated 
authorizations without needing specialized equipment! 
















1-888-646-1933 


Bumble Bee Software 
P.O. Box 2007 
Westford, MA 01886 


info@bbeesoft.com 
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The Imaging Solutions Company 






AVA TOOLS 


The Simple Yet Powerful IDE for oy n f er Lok 


Java Development Copy Protection * Electronic Softwore Distribution ¢ License Management 
Features: 


* Wraps Your Software in Minutes 

* Try-Before-You-Buy & immediate Purchase 
* Software Metering & Rentals 
es 


Help g Manual 


Writing first rate software documentation doesn't have to be 
hard. Help & Manual is a stand-alone WYSIWYG authoring 
tool, independent from your programming language and easy 
to use even for non-programmers. Write your help file once 
and create various outputs from a single source: 





















*Windows 98 HTML HELP * Printed user manuals 
* Plain HTML * Electronic manuals (RTF) 


* Help files for Windows 9x 

* Help files for Windows 3.x 

Get a fully functional evaluation copy at ec | 
http://www.ec-software.com 























> JDK Version Independent 

=> Plug-In API & Great Debugger 
> Only $59 

> Free 30-day Evaluations 





Windows & Macintosh Compatibility 


Foreign Language Support 
Key Diskette Option 
Sales: {408} 297-7444 ext.} 


www.paceap.com 
E D U C A T | O NX : a ee ee ee ee ee 


Full support for Windows98 style HTML HELP 


TOOLS 





Bayside Computing, Inc: : 
Developer Resources C, C++, and JAVA DOCUMENTATION (v. 8.0) 








































¢ Earn B.S. and M.S. in Computer Science rape Graphic-tree of Creates cross-reference 
* NEWBS in Inf tion Syst AMERICAN i 4 caller/called function hierarchy, cross- of local/global/define/parameter 
iia progian in information systems INSTITUTE eae reference, file/function index. identifiers, class trees. 
¢ Distance Education COMPUTER sc ANS 
° Object oriented B.S. program SCIENCES Ciestea/ Bisel s/MBeaIes peel ; pega integrated as 1 overall 
; blocks (functions/identifiers used) fo C-DOC program. <10,000 lines. 
7 Approved by more than 275 companies EE each function. JavTREE graphic-tree viewer (Sfree in 
¢ Follows ACM/IEEE guidelines STATE LICENSED Calculates path C-DOC) 
AND APPROVED complexity, counts lines with DOS, 
e 
Hndueal de Ohelucente RirougneunS: ACCREDITED comments, code, 'C’ statements Win95/NT, OS/2, 1,000,000+ lines. 
Free catalogue 1-800-767-AICS olelinaniints Beans, Applets, Embedded Info : Lists and action- 
of Universities dia v4 - 
. grams, or reformats source into 
or www.aics.edu and Colleges user-selected standard formats 


www.baysidecomputing.com 
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GRAPHICS LIBRARIES 


SOFTWARE BLACKSMITHS INC. email @ swbs.com 
6064 St Ives Way, Mississauga — Voice/Fax 


ONT Canada L5N-4M1 http://www.swbs.com 















PRODUCTS WANTED 


SOFTWARE PROGRAMS 
Loki for partners in profit! 
You provide the jo Small Business, SOHO, Financial, 
Personal Productivity or “how-to” software program. E-Z Legal 


Software will provide packaging, duplication, marketing and 
sales personnel to generate royalty checks for you. 


With distribution in more than 6500 retail outlets, we have 
the experience to launch, market and sell your program at retail 


at no cost to you! Dennis Liptrot: 1-800-822-4566 


384 S. Military Trail, Deerfield Beach, FL 33442 
Phone (954) 480- 8933 e Fax (954) 480-8906 
http://www.e-zlegal.com — _ dliptrot@e-zlegal.com SOFTWARE 
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VICTOR 


Image Processing Library 


Fast BMP, TIFF, PCX, GIF, TGA, PNG, JPEG. Adjust 
brightness, contrast, sharpen, create filters, resize, rotate, 
+more of single image, multiple images, or any image area; 
color reduction to optimum, specific, or std. palette; print; 
scan; crop, combine, compare, blend images. 


DOS $199, 16-bit DLL $299, 32-bit DLL $499 


Catenary Systems 
314-962-7833/fax: 314-962-8037 
www.catenary.com/victor 
ask for free demo src avail visa/mc/c.o.d. 























For more information on exhibiting, 
please contact: 

LinuxWorld Conference & Expo 

3 Speen Street, Suite 320 
Framingham, MA 01701 


For more information or to register, 
please visit: 
www.linuxworldexpo.com 

or call 800.647.1474 
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system, the power of innovation 

is in your hands! Linux users, 
softwaredevelopers, and 
businessleaders, participate in 

the evolution of the enterprise. 
LinuxWorld Conference & Expo in 
New York City is the premiere event, 
for supporters, investors and those 
interested in the linux operating 
system. Hosted in the financial 
capital of the world, the knowledge 
shared at this event will reinvent 
the way technology and 

business function 

in the new 

millennium. 


 linuxworidexpo 
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Media 
Sponsor 


THE CHRONICLE OF THE REVOLUTION 


PERFORMANCE, 


GaeRVE aS 


the journal for UNEX 
systems administrators 


Platinum Sponsors 
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SWAINE’S FLAMES 


Out Time 


onnect: Looking up host www.somewhereincyberspace.com... 
“Counting the beats,” the poet Robert Graves wrote: 


Counting the slow heart beats, 
The bleeding to death of time in slow heart beats... 


“The bleeding to death of time.” How much of my life, I wonder, is spilled as I sit staring 
through this seductive screen, waiting for a summoned web page to come to heel? 


One could, perhaps, 
Pen poems in the gaps. 


Such web-wait verse would no doubt need to be of the personal, rather than the publishable, 
variety, but no less valuable for that. 


Connect: Host www.somewhereincyberspace.com contacted. Waiting for reply... 


Odd that we have chosen to use the word valuable where valued would seem better to fit the 
intended meaning. Maybe the choice means something about the meaning; maybe language is 
telling us that we need to take note of the fact that value is something we are free to bestow, 
rather than a fixed measure assigned by the market. Perhaps one could measure web delays in 
verse forms: That was just a haiku hiatus. Yesterday I had a two-sonnet wait connecting to dub- 
dub-dub dee-dee-jay dot com. That’s nothing, this morning trying to get to dub-dub-dub linux 
dot org, I wrote an ancient Finnish epic. UNIX has been around so long that there is a rumor that 
Linus Torvalds didn’t write Linux, he just translated it from the original runes. That’s a myth, of 
course; everyone knows how Linux got started (but only Linus knows the Finnish). 

In the 1578 Peter Borne Missza edition, Rune XX of the Finnish epic the Kalevala speaks of 
time and of another fascination of Linux users: 


Time had gone but a little distance, 
Scarce a moment had passed over, 
Ere the heroes came in numbers, 
To the foaming beer of northland... 


Writer G. K. Chesterton claimed that poets do not go mad. “Mathematicians go mad...but creative 
artists very seldom.” The source I am quoting from drops Chesterton at this point like an AOL connec- 
tion, and I am left unenlightened as to whether Chesterton meant to imply that poetry kept one sane, 
or that poets are all crazy to begin with. From the poet’s point of view, perhaps it’s a distinction with- 
out a difference. Writer Stephen Harrod Buhner claims that in the craziness of alcoholic inebriation is 
the historic origin of poetry, so maybe that shines some light in the Chesterton haziness. Personally, I 
think any scholar whose definition of creative artist excludes mathematicians is a little crazy. 


Connect: Host www.somewhereincyberspace.com contacted. Waiting for reply... 


The window of my office overlooks the lawn of the restaurant of which I am part owner. As I wait 
for my web page to download, I watch the scene below. A customer fidgets, tugging at garments 
seen and unseen, as she sits in a chair under an umbrella while an apron-wearing employee bustles 
about in the sun filling orders. I, like the customer, sit waiting for my order to be filled. Of the three 
of us, the only one clearly not waiting for anything is the one called a waiter. The waiter, I muse, is 
really a server, and the waiting customer a browser. Or will be when she quits waiting because she 
has been served something to browse on. I sit at my desk fidgeting like a kid in a doctor’s waiting 
room, musing that “patient” is an amazingly inappropriate word for such sufferers. 


Patience, 7. A minor form of despair, disguised as a virtue. 
—Ambrose Bierce. 
If you’re a writer, you can only muse so long before the muse grabs you by the typing fingers; 
so I soon find myself versifying: 


There was an old man who wrote COBOL 
And gave the cold shoulder to SNOBOL, 

But about Y2K 

He had just this to say: 

In my day Running Light was thought noble. 


Connect: You have been disconnected. 
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The rest have discovered the limitations of other developer tools. 
With VisualCafé Database Edition, you’ve got all you need to get 
your job done on time and, who knows, maybe have some 
time leftover for you. Whether you’re a seasoned 
Java programmer, or moving to Java from Visual Basic® or 
C++, you'll find VisualCafé easy to learn and use because 
of its visual development environment—complete with 
automated wizards and code helper. In fact, if you’re creating 


serviets, JavaBeans,™ or 
x V4 
deploying comprehensive : 
database-driven applications Sila a e 
with JDBC support, it’s your 


best choice for development. 
It offers superior speed, robust code, drag-and-drop programming, 


QBE (Query-By-Example) capability, open Java API, and support for 
data binding JFC/Swing. Call or visit our website and request our 
Java hierarchy poster. Join the tens of thousands of developers 
who use VisualCafé. You too may develop an attachment to it. 
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